Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - SHAPE (Shape Understanding: On the Perception of Growth, Form and Process)

Teaser

How do we work out a piece of cloth is starched from the way it drapes? How do we predict how a fragile object might dent or crumple if we try to pick it up? What makes a shoe a shoe and not a fish? At first glance, these different questions might appear not to have very much...

Summary

How do we work out a piece of cloth is starched from the way it drapes? How do we predict how a fragile object might dent or crumple if we try to pick it up? What makes a shoe a shoe and not a fish? At first glance, these different questions might appear not to have very much in common. But in fact they all share have one important feature: they all involve the way we perceive and interpret shape. To work out the properties of draped cloth involves identifying and interpreting the specific shape of its creases and folds. Predicting how an object will deform when we pick it up involves recognizing the weak points in its shape and understanding how they will change in response to forces. Working out whether an object is a shoe or a fish involves identifying key features of its shape that it has in common with other members of each class. All these tasks involve perceiving, representing and interpreting the shape of objects to understand more about them. Indeed, shape perception pervades practically all aspects of visual behaviour.

There has been a lot of previous research on how the brain works out the local surface properties of shapes such as the depths or curvatures. However, practically nothing is known about how the brain uses its estimates of shape to work out other properties of objects. In SHAPE, we seek to understand how the brain goes beyond simply working out what shapes objects have, to working out their underlying ‘logic’, that is, how and why they have those shapes. We are developing a new field of study in the perceptual and cognitive sciences to unravel how the brain identifies and represents the origins of form.

Work performed

We have used a combination of approaches from psychology, neuroscience, computer graphics and machine learning to investigate how we perceive and interpret 2D and 3D shape.

We have used computer simulations and neural networks to create databases consisting of hundreds of thousands of shapes being created and altered by interactions with other objects. For example, we have simulated liquids flowing, and soft objects being squashed, as well as ‘grown’ novel animal-like outlines, using a neural network that was trained on examples of thousands of natural animal shapes. This allows us to characterize shape --- and the way shape changes --- in the natural world.

We have then used these stimuli to perform visual perception experiments on volunteers, who judge the shape, or the objects properties (e.g., how soft the object is) based on viewing the shape or watching how the shape changes over time. By carefully selecting stimuli from our giant datasets, we can tease apart the predictions of rival theories about how the brain processes and interprets shape. We then build computational models of visual perception processes.

We have used this approach to study the perception of material properties, causal history, categorization and perceived shape similarity. This approach has allowed us to show, for example, that when we view liquids, we use a combination of particular shape and motion cues to work out how viscous the liquid. These cues work across a wide range of viewing conditions.

Final results

SHAPE has significantly advanced our scientific understanding of how we visually perceive shape, which is the most important source of visual information about objects. We have created tools, methods and databases that enable the study of complex naturalistic shapes on a scale never tried previously. Among other achievements, this has resulted in a computational model for comparing how similar different shapes are, which can predict human judgments of perceived similarity for novel shapes. From now until the end of the project, we hope to be able to show how the visual system uses specific cues related to shape to estimate the material properties of objects (e.g., how soft they are) and what forces and processes have been applied to them. This will eventually lead to theoretical models that can predict human perception of shape across a wide range of conditions.