Opendata, web and dolomites


Closing the 4D Real World Reconstruction Loop

Total Cost €


EC-Contrib. €






 4DRepLy project word cloud

Explore the words cloud of the 4DRepLy project. It provides you a very rough idea of what is the project "4DRepLy" about.

deformable    motion    previously    combined    classes    difficult    capture    generality    illumination    scalability    integration    reaching    deep    crowded    virtual    revolutionary    groundbreaking    dynamic    learning    designed    diverse    methods    radically    relevance    geometry    opens    space    vision    input    plasticity    single    time    automatically    color    dense    interacting    models    first    scenes    contributions    direction    reconstruction    groups    autonomous    adapt    4dreply    generation    4d    camera    progress    model    overcomes    unbreakable    computer    complexity    occlusions    frequency    scientific    barrier    architectural    social    enhanced    reflectance    studio    loop    robotic    themselves    unlabeled    augmented    scene    visual    types    profound    handle    topology    refine    economic    builds    despite    coherent    paradigm    pervasion    grand    unprecedented    objects    interpretable    4drecon    media    fidelity    fusion    humans    shift    machine    outside    capturing    deforming    graphics    world    diversity    feasible   

Project "4DRepLy" data sheet

The following table provides information about the project.


Organization address
city: Munich
postcode: 80539

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Total cost 1˙977˙000 €
 EC max contribution 1˙977˙000 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2017-COG
 Funding Scheme ERC-COG
 Starting year 2018
 Duration (year-month-day) from 2018-09-01   to  2023-08-31


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 


 Project objective

4D reconstruction, the camera-based dense dynamic scene reconstruction, is a grand challenge in computer graphics and computer vision. Despite great progress, 4D capturing the complex, diverse real world outside a studio is still far from feasible. 4DRepLy builds a new generation of high-fidelity 4D reconstruction (4DRecon) methods. They will be the first to efficiently capture all types of deformable objects (humans and other types) in crowded real world scenes with a single color or depth camera. They capture space-time coherent deforming geometry, motion, high-frequency reflectance and illumination at unprecedented detail, and will be the first to handle difficult occlusions, topology changes and large groups of interacting objects. They automatically adapt to new scene types, yet deliver models with meaningful, interpretable parameters. This requires far reaching contributions: First, we develop groundbreaking new plasticity-enhanced model-based 4D reconstruction methods that automatically adapt to new scenes. Second, we develop radically new machine learning-based dense 4D reconstruction methods. Third, these model- and learning-based methods are combined in two revolutionary new classes of 4DRecon methods: 1) advanced fusion-based methods and 2) methods with deep architectural integration. Both, 1) and 2), are automatically designed in the 4D Real World Reconstruction Loop, a revolutionary new design paradigm in which 4DRecon methods refine and adapt themselves while continuously processing unlabeled real world input. This overcomes the previously unbreakable scalability barrier to real world scene diversity, complexity and generality. This paradigm shift opens up a new research direction in graphics and vision and has far reaching relevance across many scientific fields. It enables new applications of profound social pervasion and significant economic impact, e.g., for visual media and virtual/augmented reality, and for future autonomous and robotic systems.


year authors and title journal last update
List of publications.
2018 Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner
published pages: 96-104, ISSN: 0001-0782, DOI: 10.1145/3292039
Communications of the ACM 62/1 2020-04-15
2018 Johan Thunberg, Johan Markdahl, Florian Bernard, Jorge Goncalves
A lifting method for analyzing distributed synchronization on the unit sphere
published pages: 253-258, ISSN: 0005-1098, DOI: 10.1016/j.automatica.2018.07.007
Automatica 96 2020-04-15
2020 J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt and D. Stricker
HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Franziska Mueller, Micah Davis, Florian Bernard, Oleksandr Sotnychenko, Mickeal Verschoor, Miguel A. Otaduy, Dan Casas, Christian Theobalt
Real-time pose and shape reconstruction of two interacting hands with a single depth camera
published pages: 1-13, ISSN: 0730-0301, DOI: 10.1145/3306346.3322958
ACM Transactions on Graphics 38/4 2020-04-15
2020 M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll and C. Theobalt
DeepCap: Monocular Human Performance Capture Using Weak Supervision
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2020 Ayush Tewari, Michael Zollhofer, Florian Bernard, Pablo Garrido, Hyeongwoo Kim, Patrick Perez, Christian Theobalt
High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder
published pages: 357-370, ISSN: 0162-8828, DOI: 10.1109/tpami.2018.2876842
IEEE Transactions on Pattern Analysis and Machine Intelligence 42/2 2020-04-15
2019 Weipeng Xu, Avishek Chatterjee, Michael Zollhofer, Helge Rhodin, Pascal Fua, Hans-Peter Seidel, Christian Theobalt
Mo 2 Cap 2 : Real-time Mobile 3D Mo tion Capture with a Cap -mounted Fisheye Camera
published pages: 2093-2101, ISSN: 1077-2626, DOI: 10.1109/tvcg.2019.2898650
IEEE Transactions on Visualization and Computer Graphics 25/5 2020-04-15
2019 Florian Bernard, Johan Thunberg, Jorge Goncalves, Christian Theobalt
Synchronisation of partial multi-matchings via non-negative factorisations
published pages: 146-155, ISSN: 0031-3203, DOI: 10.1016/j.patcog.2019.03.021
Pattern Recognition 92 2020-04-15
2019 Ohad Fried, Ayush Tewari, Michael Zollhöfer, Adam Finkelstein, Eli Shechtman, Dan B Goldman, Kyle Genova, Zeyu Jin, Christian Theobalt, Maneesh Agrawala
Text-based editing of talking-head video
published pages: 1-14, ISSN: 0730-0301, DOI: 10.1145/3306346.3323028
ACM Transactions on Graphics 38/4 2020-04-15
2019 Lingjie Liu, Weipeng Xu, Michael Zollhöfer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
Neural Rendering and Reenactment of Human Actor Videos
published pages: 1-14, ISSN: 0730-0301, DOI: 10.1145/3333002
ACM Transactions on Graphics 38/5 2020-04-15
2020 Y. Zhou, M. Habermann, W. Xu, I. Habibie, C. Theobalt and F. Xu
Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Hyeongwoo Kim, Mohamed Elgharib, Michael Zollhöfer, Hans-Peter Seidel, Thabo Beeler, Christian Richardt, Christian Theobalt
Neural style-preserving visual dubbing
published pages: 1-13, ISSN: 0730-0301, DOI: 10.1145/3355089.3356500
ACM Transactions on Graphics 38/6 2020-04-15
2019 Marc Habermann, Weipeng Xu, Michael Zollhöfer, Gerard Pons-Moll, Christian Theobalt
published pages: 1-17, ISSN: 0730-0301, DOI: 10.1145/3311970
ACM Transactions on Graphics 38/2 2020-04-15
2020 Thies, Justus; Zollhöfer, Michael; Theobalt, Christian; Stamminger, Marc; Nießner, Matthias
IGNOR: Image-guided Neural Object Rendering
published pages: , ISSN: , DOI:
International Conference on Learning Representations 2020-04-15
2020 V. Golyanik and C. Theobalt
A Quantum Computational Approach to Correspondence Problems on Point Sets.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2020 A. Bozic, M. Zollhoefer, C. Theobalt and M. Niessner
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Abhimitra Meka, Christian Häne, Rohit Pandey, Michael Zollhöfer, Sean Fanello, Graham Fyffe, Adarsh Kowdle, Xueming Yu, Jay Busch, Jason Dourgarian, Peter Denny, Sofien Bouaziz, Peter Lincoln, Matt Whalen, Geoff Harvey, Jonathan Taylor, Shahram Izadi, Andrea Tagliasacchi, Paul Debevec, Christian Theobalt, Julien Valentin, Christoph Rhemann
Deep reflectance fields
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3306346.3323027
ACM Transactions on Graphics 38/4 2020-04-15
2020 A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Lingjie Liu, Nenglun Chen, Duygu Ceylan, Christian Theobalt, Wenping Wang, Niloy J. Mitra
C urve F usion
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3272127.3275097
ACM Transactions on Graphics 37/6 2020-04-15
2020 F. Bernard, Z. Khan Suri and C. Theobalt
MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Justus Thies, Michael Zollhöfer, Matthias Nießner
Deferred neural rendering
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3306346.3323035
ACM Transactions on Graphics 38/4 2020-04-15
2020 L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang and C. Theobalt
EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "4DREPLY" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "4DREPLY" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)

FICOMOL (2019)

Field Control of Cold Molecular Collisions

Read More  

Photopharm (2020)

Photopharmacology: From Academia toward the Clinic.

Read More  


The Enemy of the Good: Towards a Theory of Moral Progress

Read More