Opendata, web and dolomites


Deep Learning for Dynamic 3D Visual Scene Understanding

Total Cost €


EC-Contrib. €






Project "DeeViSe" data sheet

The following table provides information about the project.


Organization address
city: AACHEN
postcode: 52062

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Total cost 2˙000˙000 €
 EC max contribution 2˙000˙000 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2017-COG
 Funding Scheme ERC-COG
 Starting year 2018
 Duration (year-month-day) from 2018-04-01   to  2023-03-31


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 


 Project objective

Over the past 5 years, deep learning has exercised a tremendous and transformational effect on the field of computer vision. However, deep neural networks (DNNs) can only realize their full potential when applied in an end-to-end manner, i.e., when every stage of the processing pipeline is differentiable with respect to the network’s parameters, such that all of those parameters can be optimized together. Such end-to-end learning solutions are still rare for computer vision problems, in particular for dynamic visual scene understanding tasks. Moreover, feed-forward processing, as done in most DNN-based vision approaches, is only a tiny fraction of what the human brain can do. Feedback processes, temporal information processing, and memory mechanisms form an important part of our human scene understanding capabilities. Those mechanisms are currently underexplored in computer vision.

The goal of this proposal is to remove this bottleneck and to design end-to-end deep learning approaches that can realize the full potential of DNNs for dynamic visual scene understanding. We will make use of the positive interactions and feedback processes between multiple vision modalities and combine them to work towards a common goal. In addition, we will impart deep learning approaches with a notion of what it means to move through a 3D world by incorporating temporal continuity constraints, as well as by developing novel deep associative and spatial memory mechanisms.

The results of this research will enable deep neural networks to reach significantly improved dynamic scene understanding capabilities compared to today’s methods. This will have an immediate positive effect for applications in need for such capabilities, most notably for mobile robotics and intelligent vehicles.


year authors and title journal last update
List of publications.
2018 Istvan Sarandi, Timm Linder, Kai Arras, Bastian Leibe
Synthetic Occlusion Augmentation with Volumetric Heatmaps for the 2018 ECCV PoseTrack Challenge on 3D Human Pose Estimation
published pages: , ISSN: , DOI:
ECCV 2018 Workshops 2020-02-05
2018 Umer Rafi, Jürgen Gall, Bastian Leibe
Direct Shot Correspondence Matching
published pages: , ISSN: , DOI:
British Machine Vision Conference (BMVC) 2020-02-04
2019 Jonathon Luiten, Philipp Torr, Bastian Leibe
Video Instance Segmentation 2019: A Winning Approach for Combined Detection, Segmentation, Classification and Tracking
published pages: , ISSN: , DOI:
The IEEE International Conference on Computer Vision (ICCV) Workshops 2020-02-04
2019 Jonathon Luiten, Paul Voigtlaender, Bastian Leibe
Exploring the Combination of PReMVOS, BoLTVOS and UnOVOST for the 2019 YouTube-VOS Challenge
published pages: , ISSN: , DOI:
ICCV Workshops 2020-02-04
2019 Idil Esen Zülfikar, Jonathon Luiten, Bastian Leibe
UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking forthe 2019 Unsupervised DAVIS Challenge
published pages: , ISSN: , DOI:
The 2019 DAVIS Challenge on Video Object Segmentation - CVPR Workshops 2020-02-04
2018 Mahadevan, Sabarinath; Voigtlaender, Paul; Leibe, Bastian
Iteratively Trained Interactive Segmentation
published pages: , ISSN: , DOI:
British Machine Vision Conference (BMVC) 2020-02-04
2019 Jonathon Luiten, Paul Voigtlaender, Bastian Leibe
Combining PReMVOS with Box-Level Tracking for the 2019 DAVIS Challenge
published pages: , ISSN: , DOI:
The 2019 DAVIS Challenge on Video Object Segmentation - CVPR Workshops 2020-02-04
2018 Aljosa Osep, Paul Voigtlaender, Jonathon Luiten, Stefan Breuers, Bastian Leibe
Towards Large-Scale Video Video Object Mining
published pages: , ISSN: , DOI:
ECCV 2018 Workshop on Interactive and Adaptive Learning in an Open World 2020-02-04
2019 Voigtlaender, Paul; Krause, Michael; Osep, Aljosa; Luiten, Jonathon; Sekar, Berin Balachandar Gnana; Geiger, Andreas; Leibe, Bastian
MOTS: Multi-Object Tracking and Segmentation
published pages: , ISSN: , DOI:
IEEE Conference on Computer Vision and Pattern Recognition 2019-08-29
2019 Osep, Aljosa; Voigtlaender, Paul; Weber, Mark; Luiten, Jonathon; Leibe, Bastian
4D Generic Video Object Proposals
published pages: , ISSN: , DOI:
arXiv 1 2019-08-29

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "DEEVISE" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "DEEVISE" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)


The Mass Politics of Disintegration

Read More  


The Enemy of the Good: Towards a Theory of Moral Progress

Read More  


Efficient Conversion of Quantum Information Resources

Read More