#	Pagina
attuale pagina	/open-h2020/projects/210632/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - Immersify (Audiovisual Technologies for Next Generation Immersive Media)

Teaser

Summary

Virtual reality (VR) has emerged as a new form of content and experience where the user is completely involved in the content and can interact with elements of the virtual world. VR is able to provide immersive and interactive experiences that are not possible with current content formats.

As the production of immersive media content is continuously increasing, the type of platforms for consuming this new type of content is increasing as well. Not only Head-Mounted Displays (HMD) for 360Â° video are already available, but new immersive display technologies for groups of audiences are emerging including multi-display setups, immersive cinemas, and dome projections.

Several advances in media technologies are being developed in order to deliver the next generation of immersive media experiences. Some these advances include: Ultra High Definition video (8K and beyond), High Frame Rate (HFR), High Dynamic Range (HDR), Wide Color Gamut (WCG), higher Field of View, seamless interactive personalized data, Next Generation Audio (NGA), etc.
However, the video and audio compression and playback tools must also be enhanced to properly process this new immersive content. And this is the aim of Immersify: to advance the tools required for the distribution and exhibition of next-generation immersive media.

By the end of the project on March 30th 2020 we expect to achieve these goals:

1) To improve the quality of immersive media using advanced compression technology able to efficiently compress high-resolution videos, HFR, HDR, WCG, and NGA.
2) To provide media players and format conversion tools required to facilitate the consumption of immersive media in multiple display environments and multiple devices.
3) To provide tools for integrating high-quality video in seamless media presentations that will allow content creators to develop non-linear storytelling where the user has the freedom to select different paths in the content.
4) To promote immersive media content and tools in the creative and media industries in Europe.

Work performed

The work performed in the Immersify project until September 30th 2018 is summarized below:

- Encoding and streaming: Enhancements in the HEVC/H.265 encoder in terms of visual quality and encoding speed; support for partial streaming of 360Â° videos; support for the state-of-the-art ARRI omnicam format; testing the quality of the encoder with the produced 8K content and comparison with other 8K encoding solutions.
- Immersive displays: Performance optimization of the HEVC video decoder and video render engine to enable fluent playback of video formats in 8K and beyond; initial testing and integrations of the enhanced decoding and rendering capabilities into target display systems; and development of the first 16Kp60 playback system on a single workstation.
- Creative development and demonstration: Nine video clips in various technologies have been produced, three clips have been purchased from external subcontractors, and five are still in production. Story and technical description of each clips are online available in immersify.eu. Additionally, several improvements and extensions have been done for interactive media applications based on the Unity3D.
- Dissemination: The project is being promoted through the its website (immersify.eu), social media (@immersify_eu), and press releases. The developed tools have been presented at key international trade shows (NAB in Las Vegas, IBC and ISE in Amsterdam) and festivals (Festival ed Cannes and Ars Electronica Festival), among others. In one year around 350.000 people have been reached mostly professionals in the media industry. Project partners have participated regularly in cinema trade shows and discussed with their representative bodies in order to provide analyze the situation of the immersive technology and content sector and articulate mechanisms for exploitation of the developed technologies. Moreover, three technical papers on international conferences were published.
- Exploitation: Immersify has developed three main results that have been identified as exploitable foreground: codec and player tools, immersive spaces and installations, and immersive content.

Final results

The original expected results and potential impacts are:
- The main impact of Immersify will be to facilitate the adoption of the new generation of VR and immersive media by the media industry by providing tools (video compression and media playback ), creative content development tools, and guidelines for content creation, delivery and playback.
- The development of innovative solutions that will have an impact across the media value chain including: production companies, creative studios, and content exhibitors
- Tools that allow to combine ultra-high quality video with interactive elements for creating more interactive and personalized experiences.

Progress beyond the state-of-the-art
- HEVC encoder: Unlike other encoding solutions, the optimizations developed in the context of the project are addressed 1) to high-quality immersive content (8K and beyond), 2) to increase the execution time of ultra-high resolution encoding with minimal compression losses, and 3) to achieve higher robustness over a large range of bitrates and a wider variety of content.
- HEVC decoder and video render: Several tools has been implemented in the HEVC decoder and video render engine, allowing for real-time playback on a single machine of formats that were not possible to achieve before. Some of these new formats include: 16K video at 60 frames/s, 12K and 8K point cloud at 60 frames/s, stereoscopic 8Kx8K content at 60 frames/s, and very high resolution 12Kx8K 360Â° omnidirectional content.

Exploitation and use
The consortium partners are exploiting the current results of the project (performance optimized HEVC decoder and video renderer, enhanced HEVC encoder, new immersive content) making impacts in different industry areas including: museums and art festivals, 8K broadcast, 360Â° video, and large screen visualization.
Spin Digital is launching new versions of its commercial video solutions (Spin Player, Spin Enc, Spin SDK) including the enhancements being developed in the project. Spin Digital is actively promoting the new tools through demos at international trade-fairs, and by establishing partnerships with key industry stakeholders.
Ars Electronica and PSNC are using the new SDK to enhance their playback systems and the new encoder to produce high quality immersive content. First demonstrations were already performed at some of the key dissemination events including: NAB 2018, Festival de cannes 2018, Ars Electronica Festival 2018, and IBC 2018. At some of these events the consortium has presented unique demonstrations such as the first 16K video playback system presented at NAB and IBC.

Socio-economic impact and wider societal implications
Apart from the commercial exploitation, Immersify also expects to make a social impact on the audiences of the film festivals, museum, and science centers. The Deep Space installation at Ars Electronica and the dome installation at the Visualization Center C attract hundreds of thousands of visitors per year, who enjoy scientific, cultural, and artistics experiences. With the enhancements developed in this project, the user experience is becoming richer: higher quality content and more user-content interactivity.