Opendata, web and dolomites


Learning Generative 3D Scene Models for Training and Validating Intelligent Systems

Total Cost €


EC-Contrib. €






 LEGO-3D project word cloud

Explore the words cloud of the LEGO-3D project. It provides you a very rough idea of what is the project "LEGO-3D" about.

examples    arbitrary    safety    models    significantly    combine    full    entire    motion    becomes    industry    2d    generation    yielding    fidelity    geometry    entertainment    photo    viewpoints    efficient    primitives    costly    3d    data    spatio    content    scene    probabilistic    amounts    deep    recently    algorithms    capturing    primitive    artist    renderings    computer    expert    decomposition    construction    autonomous    techniques    shallow    vision    unconditional    viewpoint    synthesize    differentiable    synthesizing    representations    automate    light    extracted    creation    image    synthetic    annotated    cars    physical    transformation    invariances    away    lego    rely    material    neural    relationships    rendering    easier    designed    ways    labeled    critical    training    synthesis    networks    projections    pipeline    environments    latent    temporal    considerably    conditioned    ambiguities    generative    hard    tackle    variety    manipulation    automatic    collecting    learning    occlusion    representation    devise    jointly    simulation    validation    realistic    scenes    captured    simulators    augmentation   

Project "LEGO-3D" data sheet

The following table provides information about the project.


Organization address
postcode: 72074

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country Germany [DE]
 Total cost 1˙467˙500 €
 EC max contribution 1˙467˙500 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2019-STG
 Funding Scheme ERC-STG
 Starting year 2020
 Duration (year-month-day) from 2020-10-01   to  2025-09-30


Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 


 Project objective

Recently, the field of computer vision has witnessed a major transformation away from expert designed shallow models towards more generic deep representation learning. However, collecting labeled data for training deep models is costly and existing simulators with artist-designed scenes do not provide the required variety and fidelity. Project LEGO-3D will tackle this problem by developing probabilistic models capable of synthesizing 3D scenes jointly with photo-realistic 2D projections from arbitrary viewpoints and with full control over the scene elements. Our key insight is that data augmentation, while hard in 2D, becomes considerably easier in 3D as physical properties such as viewpoint invariances and occlusion relationships are captured by construction. Thus, our goal is to learn the entire 3D-to-2D simulation pipeline. In particular, we will focus on the following problems:

(A) We will devise algorithms for automatic decomposition of real and synthetic scenes into latent 3D primitive representations capturing geometry, material, light and motion.

(B) We will develop novel probabilistic generative models which are able to synthesize large-scale 3D environments based on the primitives extracted in project (A). In particular, we will develop unconditional, conditioned and spatio-temporal scene generation networks.

(C) We will combine differentiable and neural rendering techniques with deep learning based image synthesis, yielding high-fidelity 2D renderings of the 3D representations generated in project (B) while capturing ambiguities and uncertainties.

Project LEGO-3D will significantly impact a large number of application areas. Examples include vision systems which require access to large amounts of annotated data, safety-critical applications such as autonomous cars that rely on efficient ways for training and validation, as well as the entertainment industry which seeks to automate the creation and manipulation of 3D content.

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "LEGO-3D" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email ( and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "LEGO-3D" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)


The Enemy of the Good: Towards a Theory of Moral Progress

Read More  

SUExp (2018)

Strategic Uncertainty: An Experimental Investigation

Read More  


The Mass Politics of Disintegration

Read More