Opendata, web and dolomites

4DRepLy SIGNED

Closing the 4D Real World Reconstruction Loop

Total Cost €

0

EC-Contrib. €

0

Partnership

0

Views

0

 4DRepLy project word cloud

Explore the words cloud of the 4DRepLy project. It provides you a very rough idea of what is the project "4DRepLy" about.

deformable    builds    capturing    occlusions    unprecedented    relevance    studio    automatically    vision    color    architectural    dynamic    computer    diverse    unbreakable    learning    4dreply    paradigm    interacting    previously    direction    contributions    time    refine    space    generality    radically    barrier    shift    models    diversity    fusion    loop    augmented    reconstruction    classes    first    integration    despite    capture    world    grand    4d    handle    illumination    dense    methods    plasticity    combined    revolutionary    geometry    groups    types    feasible    robotic    machine    camera    themselves    reflectance    deforming    outside    scalability    generation    economic    adapt    enhanced    graphics    social    topology    crowded    overcomes    virtual    unlabeled    motion    designed    profound    progress    scene    opens    pervasion    frequency    deep    interpretable    visual    fidelity    input    coherent    reaching    difficult    objects    4drecon    complexity    scenes    humans    scientific    groundbreaking    single    media    autonomous    model   

Project "4DRepLy" data sheet

The following table provides information about the project.

Coordinator
MAX-PLANCK-GESELLSCHAFT ZUR FORDERUNG DER WISSENSCHAFTEN EV 

Organization address
address: HOFGARTENSTRASSE 8
city: Munich
postcode: 80539
website: www.mpg.de

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Total cost 1˙977˙000 €
 EC max contribution 1˙977˙000 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2017-COG
 Funding Scheme ERC-COG
 Starting year 2018
 Duration (year-month-day) from 2018-09-01   to  2023-08-31

 Partnership

Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 
1    MAX-PLANCK-GESELLSCHAFT ZUR FORDERUNG DER WISSENSCHAFTEN EV DE (Munich) coordinator 1˙977˙000.00

Map

 Project objective

4D reconstruction, the camera-based dense dynamic scene reconstruction, is a grand challenge in computer graphics and computer vision. Despite great progress, 4D capturing the complex, diverse real world outside a studio is still far from feasible. 4DRepLy builds a new generation of high-fidelity 4D reconstruction (4DRecon) methods. They will be the first to efficiently capture all types of deformable objects (humans and other types) in crowded real world scenes with a single color or depth camera. They capture space-time coherent deforming geometry, motion, high-frequency reflectance and illumination at unprecedented detail, and will be the first to handle difficult occlusions, topology changes and large groups of interacting objects. They automatically adapt to new scene types, yet deliver models with meaningful, interpretable parameters. This requires far reaching contributions: First, we develop groundbreaking new plasticity-enhanced model-based 4D reconstruction methods that automatically adapt to new scenes. Second, we develop radically new machine learning-based dense 4D reconstruction methods. Third, these model- and learning-based methods are combined in two revolutionary new classes of 4DRecon methods: 1) advanced fusion-based methods and 2) methods with deep architectural integration. Both, 1) and 2), are automatically designed in the 4D Real World Reconstruction Loop, a revolutionary new design paradigm in which 4DRecon methods refine and adapt themselves while continuously processing unlabeled real world input. This overcomes the previously unbreakable scalability barrier to real world scene diversity, complexity and generality. This paradigm shift opens up a new research direction in graphics and vision and has far reaching relevance across many scientific fields. It enables new applications of profound social pervasion and significant economic impact, e.g., for visual media and virtual/augmented reality, and for future autonomous and robotic systems.

 Publications

year authors and title journal last update
List of publications.
2018 Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner
Face2Face
published pages: 96-104, ISSN: 0001-0782, DOI: 10.1145/3292039
Communications of the ACM 62/1 2020-04-15
2018 Johan Thunberg, Johan Markdahl, Florian Bernard, Jorge Goncalves
A lifting method for analyzing distributed synchronization on the unit sphere
published pages: 253-258, ISSN: 0005-1098, DOI: 10.1016/j.automatica.2018.07.007
Automatica 96 2020-04-15
2020 J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt and D. Stricker
HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Franziska Mueller, Micah Davis, Florian Bernard, Oleksandr Sotnychenko, Mickeal Verschoor, Miguel A. Otaduy, Dan Casas, Christian Theobalt
Real-time pose and shape reconstruction of two interacting hands with a single depth camera
published pages: 1-13, ISSN: 0730-0301, DOI: 10.1145/3306346.3322958
ACM Transactions on Graphics 38/4 2020-04-15
2020 M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll and C. Theobalt
DeepCap: Monocular Human Performance Capture Using Weak Supervision
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2020 Ayush Tewari, Michael Zollhofer, Florian Bernard, Pablo Garrido, Hyeongwoo Kim, Patrick Perez, Christian Theobalt
High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder
published pages: 357-370, ISSN: 0162-8828, DOI: 10.1109/tpami.2018.2876842
IEEE Transactions on Pattern Analysis and Machine Intelligence 42/2 2020-04-15
2019 Weipeng Xu, Avishek Chatterjee, Michael Zollhofer, Helge Rhodin, Pascal Fua, Hans-Peter Seidel, Christian Theobalt
Mo 2 Cap 2 : Real-time Mobile 3D Mo tion Capture with a Cap -mounted Fisheye Camera
published pages: 2093-2101, ISSN: 1077-2626, DOI: 10.1109/tvcg.2019.2898650
IEEE Transactions on Visualization and Computer Graphics 25/5 2020-04-15
2019 Florian Bernard, Johan Thunberg, Jorge Goncalves, Christian Theobalt
Synchronisation of partial multi-matchings via non-negative factorisations
published pages: 146-155, ISSN: 0031-3203, DOI: 10.1016/j.patcog.2019.03.021
Pattern Recognition 92 2020-04-15
2019 Ohad Fried, Ayush Tewari, Michael Zollhöfer, Adam Finkelstein, Eli Shechtman, Dan B Goldman, Kyle Genova, Zeyu Jin, Christian Theobalt, Maneesh Agrawala
Text-based editing of talking-head video
published pages: 1-14, ISSN: 0730-0301, DOI: 10.1145/3306346.3323028
ACM Transactions on Graphics 38/4 2020-04-15
2019 Lingjie Liu, Weipeng Xu, Michael Zollhöfer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
Neural Rendering and Reenactment of Human Actor Videos
published pages: 1-14, ISSN: 0730-0301, DOI: 10.1145/3333002
ACM Transactions on Graphics 38/5 2020-04-15
2020 Y. Zhou, M. Habermann, W. Xu, I. Habibie, C. Theobalt and F. Xu
Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Hyeongwoo Kim, Mohamed Elgharib, Michael Zollhöfer, Hans-Peter Seidel, Thabo Beeler, Christian Richardt, Christian Theobalt
Neural style-preserving visual dubbing
published pages: 1-13, ISSN: 0730-0301, DOI: 10.1145/3355089.3356500
ACM Transactions on Graphics 38/6 2020-04-15
2019 Marc Habermann, Weipeng Xu, Michael Zollhöfer, Gerard Pons-Moll, Christian Theobalt
LiveCap
published pages: 1-17, ISSN: 0730-0301, DOI: 10.1145/3311970
ACM Transactions on Graphics 38/2 2020-04-15
2020 Thies, Justus; Zollhöfer, Michael; Theobalt, Christian; Stamminger, Marc; Nießner, Matthias
IGNOR: Image-guided Neural Object Rendering
published pages: , ISSN: , DOI:
International Conference on Learning Representations 2020-04-15
2020 V. Golyanik and C. Theobalt
A Quantum Computational Approach to Correspondence Problems on Point Sets.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2020 A. Bozic, M. Zollhoefer, C. Theobalt and M. Niessner
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Abhimitra Meka, Christian Häne, Rohit Pandey, Michael Zollhöfer, Sean Fanello, Graham Fyffe, Adarsh Kowdle, Xueming Yu, Jay Busch, Jason Dourgarian, Peter Denny, Sofien Bouaziz, Peter Lincoln, Matt Whalen, Geoff Harvey, Jonathan Taylor, Shahram Izadi, Andrea Tagliasacchi, Paul Debevec, Christian Theobalt, Julien Valentin, Christoph Rhemann
Deep reflectance fields
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3306346.3323027
ACM Transactions on Graphics 38/4 2020-04-15
2020 A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H-P. Seidel, P. Perez, M. Zollhöfer and C. Theobalt
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Lingjie Liu, Nenglun Chen, Duygu Ceylan, Christian Theobalt, Wenping Wang, Niloy J. Mitra
C urve F usion
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3272127.3275097
ACM Transactions on Graphics 37/6 2020-04-15
2020 F. Bernard, Z. Khan Suri and C. Theobalt
MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment.
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15
2019 Justus Thies, Michael Zollhöfer, Matthias Nießner
Deferred neural rendering
published pages: 1-12, ISSN: 0730-0301, DOI: 10.1145/3306346.3323035
ACM Transactions on Graphics 38/4 2020-04-15
2020 L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang and C. Theobalt
EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera
published pages: , ISSN: , DOI:
2020 IEEE/CVF International Conference on Computer Vision 2020-04-15

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "4DREPLY" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email (fabio@fabiodisconzi.com) and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "4DREPLY" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)

CohoSing (2019)

Cohomology and Singularities

Read More  

ZARAH (2020)

Women’s labour activism in Eastern Europe and transnationally, from the age of empires to the late 20th century

Read More  

MOCHA (2019)

Understanding and leveraging ‘moments of change’ for pro-environmental behaviour shifts

Read More