Opendata, web and dolomites


Closing the 4D Real World Reconstruction Loop

Total Cost €


EC-Contrib. €






 4DRepLy project word cloud

Explore the words cloud of the 4DRepLy project. It provides you a very rough idea of what is the project "4DRepLy" about.

unlabeled    illumination    outside    groundbreaking    designed    4drecon    profound    computer    contributions    unprecedented    overcomes    space    plasticity    scalability    motion    media    models    reconstruction    feasible    relevance    frequency    classes    vision    camera    reflectance    diverse    interpretable    capturing    graphics    single    learning    capture    previously    loop    augmented    architectural    topology    barrier    methods    deformable    types    complexity    adapt    4d    visual    scene    time    deep    virtual    world    machine    deforming    opens    reaching    generality    refine    studio    autonomous    generation    progress    interacting    model    themselves    automatically    fusion    dense    economic    robotic    grand    humans    difficult    geometry    scenes    scientific    unbreakable    4dreply    despite    combined    color    handle    radically    fidelity    first    builds    crowded    direction    revolutionary    coherent    pervasion    integration    enhanced    paradigm    social    objects    dynamic    occlusions    input    groups    diversity    shift   

Project "4DRepLy" data sheet

The following table provides information about the project.


Organization address
city: Munich
postcode: 80539

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Total cost 1˙977˙000 €
 EC max contribution 1˙977˙000 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2017-COG
 Funding Scheme ERC-COG
 Starting year 2018
 Duration (year-month-day) from 2018-09-01   to  2023-08-31


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 


 Project objective

4D reconstruction, the camera-based dense dynamic scene reconstruction, is a grand challenge in computer graphics and computer vision. Despite great progress, 4D capturing the complex, diverse real world outside a studio is still far from feasible. 4DRepLy builds a new generation of high-fidelity 4D reconstruction (4DRecon) methods. They will be the first to efficiently capture all types of deformable objects (humans and other types) in crowded real world scenes with a single color or depth camera. They capture space-time coherent deforming geometry, motion, high-frequency reflectance and illumination at unprecedented detail, and will be the first to handle difficult occlusions, topology changes and large groups of interacting objects. They automatically adapt to new scene types, yet deliver models with meaningful, interpretable parameters. This requires far reaching contributions: First, we develop groundbreaking new plasticity-enhanced model-based 4D reconstruction methods that automatically adapt to new scenes. Second, we develop radically new machine learning-based dense 4D reconstruction methods. Third, these model- and learning-based methods are combined in two revolutionary new classes of 4DRecon methods: 1) advanced fusion-based methods and 2) methods with deep architectural integration. Both, 1) and 2), are automatically designed in the 4D Real World Reconstruction Loop, a revolutionary new design paradigm in which 4DRecon methods refine and adapt themselves while continuously processing unlabeled real world input. This overcomes the previously unbreakable scalability barrier to real world scene diversity, complexity and generality. This paradigm shift opens up a new research direction in graphics and vision and has far reaching relevance across many scientific fields. It enables new applications of profound social pervasion and significant economic impact, e.g., for visual media and virtual/augmented reality, and for future autonomous and robotic systems.


year authors and title journal last update
List of publications.
2018 Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner
published pages: 96-104, ISSN: 0001-0782, DOI: 10.1145/3292039
Communications of the ACM 62/1 2020-04-15
2018 Johan Thunberg, Johan Markdahl, Florian Bernard, Jorge Goncalves
A lifting method for analyzing distributed synchronization on the unit sphere
published pages: 253-258, ISSN: 0005-1098, DOI: 10.1016/j.automatica.2018.07.007
Automatica 96 2020-04-15
2020 J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt and D. Stricker
HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Franziska Mueller, Micah Davis, Florian Bernard, Oleksandr Sotnychenko, Mickeal Verschoor, Miguel A. Otaduy, Dan Casas, Christian Theobalt
Real-time pose and shape reconstruction of two interacting hands with a single depth camera
published pages: 1-13, ISSN: 0730-0301, DOI: 10.1145/3306346.3322958
ACM Transactions on Graphics 38/4 2020-04-15
2020 M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll and C. Theobalt
DeepCap: Monocular Human Performance Capture Using Weak Supervision
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2020 Ayush Tewari, Michael Zollhofer, Florian Bernard, Pablo Garrido, Hyeongwoo Kim, Patrick Perez, Christian Theobalt
High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder
published pages: 357-370, ISSN: 0162-8828, DOI: 10.1109/tpami.2018.2876842
IEEE Transactions on Pattern Analysis and Machine Intelligence 42/2 2020-04-15
2019 Weipeng Xu, Avishek Chatterjee, Michael Zollhofer, Helge Rhodin, Pascal Fua, Hans-Peter Seidel, Christian Theobalt
Mo 2 Cap 2 : Real-time Mobile 3D Mo tion Capture with a Cap -mounted Fisheye Camera
published pages: 2093-2101, ISSN: 1077-2626, DOI: 10.1109/tvcg.2019.2898650
IEEE Transactions on Visualization and Computer Graphics 25/5 2020-04-15
2019 Florian Bernard, Johan Thunberg, Jorge Goncalves, Christian Theobalt
Synchronisation of partial multi-matchings via non-negative factorisations
published pages: 146-155, ISSN: 0031-3203, DOI: 10.1016/j.patcog.2019.03.021
Pattern Recognition 92 2020-04-15
2019 Ohad Fried, Ayush Tewari, Michael Zollhöfer, Adam Finkelstein, Eli Shechtman, Dan B Goldman, Kyle Genova, Zeyu Jin, Christian Theobalt, Maneesh Agrawala
Text-based editing of talking-head video
published pages: 1-14, ISSN: 0730-0301, DOI: 10.1145/3306346.3323028
ACM Transactions on Graphics 38/4 2020-04-15
2019 Lingjie Liu, Weipeng Xu, Michael Zollhöfer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
Neural Rendering and Reenactment of Human Actor Videos
published pages: 1-14, ISSN: 0730-0301, DOI: 10.1145/3333002
ACM Transactions on Graphics 38/5 2020-04-15
2020 Y. Zhou, M. Habermann, W. Xu, I. Habibie, C. Theobalt and F. Xu
Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Hyeongwoo Kim, Mohamed Elgharib, Michael Zollhöfer, Hans-Peter Seidel, Thabo Beeler, Christian Richardt, Christian Theobalt
Neural style-preserving visual dubbing
published pages: 1-13, ISSN: 0730-0301, DOI: 10.1145/3355089.3356500
ACM Transactions on Graphics 38/6 2020-04-15
2019 Marc Habermann, Weipeng Xu, Michael Zollhöfer, Gerard Pons-Moll, Christian Theobalt
published pages: 1-17, ISSN: 0730-0301, DOI: 10.1145/3311970
ACM Transactions on Graphics 38/2 2020-04-15
2020 Thies, Justus; Zollhöfer, Michael; Theobalt, Christian; Stamminger, Marc; Nießner, Matthias
IGNOR: Image-guided Neural Object Rendering
published pages: , ISSN: , DOI:
International Conference on Learning Representations 2020-04-15
2020 V. Golyanik and C. Theobalt
A Quantum Computational Approach to Correspondence Problems on Point Sets.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2020 A. Bozic, M. Zollhoefer, C. Theobalt and M. Niessner
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Abhimitra Meka, Christian Häne, Rohit Pandey, Michael Zollhöfer, Sean Fanello, Graham Fyffe, Adarsh Kowdle, Xueming Yu, Jay Busch, Jason Dourgarian, Peter Denny, Sofien Bouaziz, Peter Lincoln, Matt Whalen, Geoff Harvey, Jonathan Taylor, Shahram Izadi, Andrea Tagliasacchi, Paul Debevec, Christian Theobalt, Julien Valentin, Christoph Rhemann
Deep reflectance fields
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3306346.3323027
ACM Transactions on Graphics 38/4 2020-04-15
2020 A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Lingjie Liu, Nenglun Chen, Duygu Ceylan, Christian Theobalt, Wenping Wang, Niloy J. Mitra
C urve F usion
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3272127.3275097
ACM Transactions on Graphics 37/6 2020-04-15
2020 F. Bernard, Z. Khan Suri and C. Theobalt
MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Justus Thies, Michael Zollhöfer, Matthias Nießner
Deferred neural rendering
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3306346.3323035
ACM Transactions on Graphics 38/4 2020-04-15
2020 L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang and C. Theobalt
EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "4DREPLY" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "4DREPLY" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)

CohoSing (2019)

Cohomology and Singularities

Read More  

EAST (2020)

Using Evolutionary Algorithms to Understand and Secure Web/Enterprise Systems

Read More  

DINAMIX (2019)

Real-time diffusion NMR analysis of mixtures

Read More