Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - V4Design (Visual and textual content re-purposing FOR(4) architecture, Design and video virtual reality games)

Teaser

Today, vast troves of visual and textual data, which are of great interest to architects and video game designers, such as paintings, archival footage, documentaries, movies, reviews or catalogues, and various other forms of artwork, are currently difficult to access and...

Summary

Today, vast troves of visual and textual data, which are of great interest to architects and video game designers, such as paintings, archival footage, documentaries, movies, reviews or catalogues, and various other forms of artwork, are currently difficult to access and cannot be easily incorporated in the design process. Such works of art can serve as sources of inspiration and design direction towards innovative designs, new concepts, or period-focused structures, among others.

V4Design aims to enable the re-use and re-purpose of such multimedia content by proposing an innovative solution for its effective integration, so as to extract 3D representations and VR game environments. This will allow architects, designers and video game creators to re-use heterogeneous archives of already available and retrieved digital content and re-purpose it by making the wealth of 3D, VR, aesthetic and textual information easily accessible and providing resources and tools to design and model outdoors and indoors environments of architecture and VR video game projects.

To achieve this, V4Design builds upon the concept of semantic integration of heterogeneous 2-dimensional multimedia in order to generate enhanced dynamic 3-dimensional structures and environments. Towards this direction V4Design collects data from the consortium partners, and the web by crawling online art libraries and databases for freely available content (textual and visual). The collected content is used to create dynamic 3D-models of objects, buildings/structures and to support 3D and VR environments. Architects and video game designers are thus able to easily access and use a realistic and comprehensive representation of structures of interest from various time periods and styles in an economical and cost-effective manner. Innovative results that integrate past aesthetic trends, localization of buildings and art-elements in visual data, text generation and semantic knowledge is also produced as complementary material to support the design process.

In a nutshell, V4Design goal is to: (1) re-use textual and visual content by enabling its efficient collection from content providers and crawling from public web resources; (2) re-purpose of content by developing novel approaches for 3D reconstruction and modelling, buildings and objects localization, aesthetics and style extraction, generation of 3D objects enhanced with semantics and explanatory text descriptions and finally (3) deploy innovative architecture, design and VR game authoring applications.

Work performed

During the first half (M1-M18) of the project, V4Design achieved all the objectives and reached all milestones set for this reporting period. More specifically, in the first year of the project (M1-M12), empirical studies of the relevant material were performed and the operational infrastructure has been set up for all modules. The user requirements have been defined (MS1), based on which the V4Design technical specifications have been set and the platform architecture has been designed. This architecture was realised by an operational prototype (MS2), which integrates the skeleton versions of the research modules, and served as the basis towards the first prototype (MS3).

In the first half of the second year (M13-M18) and towards MS3, research and development have been continued and several scientific, technological, user and impact creation objectives have been achieved. More specifically, the first versions of all components for a) 3D- Reconstruction and Visual Analysis, b) Content Extraction from Textual and Visual Data, c) Semantic Content Representation, Integration and Reasoning, and d) Multilingual Content Summarization have been delivered and integrated into the first prototype of the V4Design platform. At the same time, the V4Design data repository has been enriched with digital content from the Web but also with content provided by the consortium content providers. Valuable feedback and suggestions have been elicited from the users about the collected data, usability and functionality of the authoring tools, as well as preliminary feedback on the platform results that guided the research and development activities. Dissemination and collaboration activities have further progressed, expanding the user groups and participation in relevant events. Finally, updated market analysis, exploitation plan and business models have been delivered.

Final results

In a world where visual and textual data are in abundance, V4Design is dedicated towards the re-use and re-purpose of digital content, aiming at supporting central players in the creative industries, namely architects, game designers, as well as designers of any kind in their workflow.

V4Design will achieve the following impacts:
•Re-use and re-purpose the vast troves of existing digital content.
•Improve the inspiration and design process.
•Assist architects, designers and video game creators to have a greater impact with their work.

The expected final results in an integrated and flexible platform that will revolve around two main axes: (i) design of architectural structures for exterior (Use Case 1) and exterior purposes (Use Case 2); (ii) creation of immersive VR environments for exterior (Use Case 4) and interior spaces (Use Case 3). V4Design pertain to the listed scientific fields and will facilitate the implementation of the promised impact:
•Data acquisition: V4Design will provide a collection platform that will enable the acquisition of visual and their related textual context. Data will be either obtained from consortium data providers or crawled /scraped from renowned art-related websites and domains.
•Textual understanding: V4Design will integrate innovative knowledge-based solutions into existing dependency-based natural language understanding systems to support deeper and more robust analysis of textual content.
•Visual understanding: V4Design will analyze famous paintings and other artwork depictions in order to acquire their aesthetics/stylistic representation and their consequent emotion stimulation to the viewer. Sophisticated categorization will follow up so as to enable end-users to retrieve and create the texture that they want to deploy in their creation.
•Building detection and localization: V4Design will analyze the visual content of videos in order to detect objects and specific rooms from the interior of buildings and viewpoints from their exterior. This will enable the user to search and retrieve specific places and buildings throughout a vast amount of visual data and use them to extract their 3D models or get inspired.
•3D reconstruction, 3D modeling and BIM/GIS enhancement: V4Design will enable the analysis of interior design objects and multiple images from the exterior of buildings in order to extract their 3D BIM models.
•Semantics Integration, Fusion and Retrieval: V4Design models will deploy and extend existing ontologies to represent all aspects pertinent to the project and to support the end-users’ needs in real use case scenarios. V4Design will also apply innovative, user-tailored and synergistic combinations of statistics-based, rule and knowledge-based solutions. A Linked Data approach will follow up so as to enable the combining of a wide variety of information sources with constructed 3D models

Website & more info

More info: https://v4design.eu/.