Opendata, web and dolomites

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - MoTIVE (Moments in Time in Immersive Virtual Environments)

Teaser

We all have precious moments in our lives that could be enjoyable to relive. Additionally, there are probably for most people events that happened in the past that they were unable to attend, but would have liked to be able to do so, and still regret the lost opportunity. The...

Summary

We all have precious moments in our lives that could be enjoyable to relive. Additionally, there are probably for most people events that happened in the past that they were unable to attend, but would have liked to be able to do so, and still regret the lost opportunity. The idea of MoTIVE is to be able to reconstruct special moments from the past making use of photographs and video, and let people experience those moments as if they were there – exploiting the technology of immersive virtual reality (VR). In particular the project focusses on music concerts from the 1980s and 1990s, that people can experience in many different ways.

The research in the project involves both technical and scientific advances. When you attend a concert it is together with thousands of other people. Part of our technical research is the construction of Agent Based Models of an audience moving towards the concert venue, finding their way through the building to their seats. The human participant also has to navigate to the concert. In VR it is possible to physically walk, although the space in which you can walk is typically much smaller than the virtual space that is represented. For example, you can be tracked in a small room-sized space, and physically walk through it, but of course the concert venue is very much larger than the physical space in which you are located. How are you able to make your way through the virtual space? Hence the project is investigating different paradigms for locomoting through a virtual space that is much larger than the physical space in which the VR system is embedded. However, simulator sickness may occur when moving through a virtual space when your visual system receives information that you are moving, but this is not accompanied by the usual sensations of moving, such as the proprioceptive and vestibular signals associated with movement. The MoTIVE project is investigating a number of methods to reduce the probability of this sickness occurring.

When you attend the virtual concert will you have the sensation of presence - being there, with the events depicted seemingly real. The scientific side of the research will focus on how to maximise the probability of presence occurring, and also how to measure these subjective sensations.

How are you represented in VR when you attend the concert? Here we continue our long-standing research on body ownership illusions and their consequences. Body ownership is the illusion that the life-sized virtual body that you see when you look down towards yourself in VR, or when you see it in a reflected surface feels as though it is your body. It has been shown that this illusion occurs and also has consequences. For example, when embodied in a body that resembles Albert Einstein people do better on a cognitive test than when embodied in a neutral body. In MoTIVE we are looking at what we have called ‘cognitive embodiment’ – which is the impact that the virtual situation has on the level of body ownership and its consequences.

Finally, how is it possible to reconstruct a version of a concert in VR? Here we are working with photographs and videos that are available. Given a photograph of a musician, for example, it is possible to extract sufficient information to make a 3D model of the face and body, and given a video it is possible to extract the movements of people. We are using these methods to reconstruct in 3D a representation of the concert that can be used for the VR scenario.

The overall objectives are therefore:

• Construct VR scenarios of past events especially old music concerts.
• Populate the concerts with thousands of virtual characters representing the audience.
• Embed the human participant in the scenario as a member of the audience.
• Investigate how the level of presence (Place Illusion and Plausibility) varies with different representations and interactive techniques, and measures of these concepts.
• Use the scenario to study b

Work performed

We have built a first version of the ABM system to represent crowds (for example, attending a concert). The ABM supports crowds of thousands of individual agents. that can avoid each other, walk in groups, talk one to another, look at each other or react to certain events (for example they would applause when the concert starts). Our ABM also presents an emotion contagion system, where some of the psychological parameters variable through time can be spread through the crowd, such as members of the crowd being influenced by one character standing up and dancing.

In relation to measurement of participant experience in VR, a new paradigm using Machine Learning has been invented and used in an experimental study. It has been integrated into an initial concert scenario.

With respect to body ownership and its consequences we have found that the social context of the embodiment influences both the level of embodiment, and the subsequent effects of embodiment, an experiment in the context of racial bias.

A walking in place method for navigation through large virtual environments has been integrated into the concert scenario that was used for the Machine Learning based experiment. In the experimental study participants found that the walking in place method led to a greater sense of realism, but was more tiring.

We have developed a complete pipeline to obtain the 3D reconstruction and animation associated to each character in a musical performance, and we have worked in combining several 2D frames to enhance the 3D volumetric reconstruction of the characters. All these improvements will be published in international conferences and journals.

Final results

By the end of the project we expect participants will be able to attend virtual concerts, reconstructed from historical (video and photographic evidence) and have the strong feeling of being there in real events, and be able to experience the concert from different points of view (in the audience, on the stage, as a musician, etc). We will have made important discoveries about presence in VR, and also new methods for measurement. Similarly we will have made new discoveries in the area of body ownership. We aim to improve methods of navigation in VR, and diminish the probability of simulator sickness.

Website & more info

More info: http://www.motiverc.org.