The neural basis of depth perception from motion parallax

In addition to depth cues afforded by binocular vision, the brain processes relative motion signals to perceive depth. When an observer translates relative to their visual environment, the relative motion of objects at different distances (motion parallax) provides a powerful cue to three-dimensional scene structure. Although perception of depth based on motion parallax has been studied extensively in humans, relatively little is known regarding the neural basis of this visual capability. We review recent advances in elucidating the neural mechanisms for representing depth-sign (near versus far) from motion parallax. We examine a potential neural substrate in the middle temporal visual area for depth perception based on motion parallax, and we explore the nature of the signals that provide critical inputs for disambiguating depth-sign. This article is part of the themed issue ‘Vision in our three-dimensional world’.

[1]  Mark Nawrot,et al.  Eye movements provide the extra-retinal signal required for the perception of depth from motion parallax , 2003, Vision Research.

[2]  I. Howard,et al.  Seeing in depth, Vol. 2: Depth perception. , 2002 .

[3]  Dora E Angelaki,et al.  Joint Representation of Depth from Motion Parallax and Binocular Disparity Cues in Macaque Area MT , 2013, The Journal of Neuroscience.

[4]  Gregory C DeAngelis,et al.  Disparity Channels in Early Vision , 2007, The Journal of Neuroscience.

[5]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[6]  B JULESZ,et al.  Binocular Depth Perception without Familiarity Cues , 1964, Science.

[7]  Dora E Angelaki,et al.  Role of visual and non-visual cues in constructing a rotation-invariant representation of heading in parietal cortex , 2015, eLife.

[8]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[9]  Mark Nawrot,et al.  MT Neurons Combine Visual Motion with a Smooth Eye Movement Signal to Code Depth-Sign from Motion Parallax , 2009, Neuron.

[10]  John H. R. Maunsell,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. , 1983, Journal of neurophysiology.

[11]  Dora E Angelaki,et al.  A Functional Link between MT Neurons and Depth Perception Based on Motion Parallax , 2015, The Journal of Neuroscience.

[12]  Dora E Angelaki,et al.  Functional Specializations of the Ventral Intraparietal Area for Multisensory Heading Discrimination , 2013, The Journal of Neuroscience.

[13]  Peter H. Schiller,et al.  The integration of disparity, shading and motion parallax cues for depth perception in humans and monkeys , 2011, Brain Research.

[14]  G. DeAngelis,et al.  Organization of Disparity-Selective Neurons in Macaque Area MT , 1999, The Journal of Neuroscience.

[15]  S. McKee,et al.  The precision of binocular and monocular depth judgments in natural settings. , 2010, Journal of vision.

[16]  Hiroshi Ono,et al.  Depth and Motion in Historical Descriptions of Motion Parallax , 2005, Perception.

[17]  Curtis L Baker,et al.  Contribution of motion parallax to segmentation and depth perception. , 2011, Journal of vision.

[18]  Alexandre Pouget,et al.  Computational approaches to sensorimotor transformations , 2000, Nature Neuroscience.

[19]  G. DeAngelis,et al.  Linking Neural Representation to Function in Stereoscopic Depth Perception: Roles of the Middle Temporal Area in Coarse versus Fine Disparity Discrimination , 2006, The Journal of Neuroscience.

[20]  J. Movshon,et al.  Modulation of visual signals in macaque MT and MST neurons during pursuit eye movement. , 2009, Journal of Neurophysiology.

[21]  A retrospective. , 2007, Ostomy/wound management.

[22]  Lindsey Joyce,et al.  The pursuit theory of motion parallax , 2006, Vision Research.

[23]  H. Ono,et al.  Apparent Depth with Motion Aftereffect and Head Movement , 1994, Perception.

[24]  G C DeAngelis,et al.  The physiology of stereopsis. , 2001, Annual review of neuroscience.

[25]  Raymond Campan,et al.  The visual perception of relative distances in the wood‐cricket, Nemobius sylvestris , 1981 .

[26]  E. C. Sobel.  Depth perception by motion parallax and paradoxical parallax in the locust , 2005, Naturwissenschaften.

[27]  J. Farber,et al.  Optical motions as information for unsigned depth. , 1979, Journal of experimental psychology. Human perception and performance.

[28]  B. Rogers,et al.  The Interaction of Binocular Disparity and Motion Parallax in the Computation of Depth , 1996, Vision Research.

[29]  B Rogers,et al.  Motion Parallax as an Independent Cue for Depth Perception , 1979, Perception.

[30]  R. Born,et al.  Integrating motion and depth via parallel pathways , 2008, Nature Neuroscience.

[31]  G. DeAngelis,et al.  Contribution of Area MT to Stereoscopic Depth Perception Choice-Related Response Modulations Reflect Task Strategy , 2004, Neuron.

[32]  Fulvio Domini,et al.  Stereo and motion information are not independently processed by the visual system , 2006, Vision Research.

[33]  Gregory C. DeAngelis,et al.  A neural representation of depth from motion parallax in macaque visual cortex , 2008, Nature.

[34]  Jenny J. Naji,et al.  Perceiving depth order during pursuit eye movement , 2004, Vision Research.

[35]  An Cao,et al.  Behavioral assessment of motion parallax and stereopsis as depth cues in rhesus monkeys , 2002, Vision Research.

[36]  Takahisa M. Sanada,et al.  Representation of 3-D surface orientation by velocity and disparity gradient cues in area MT. , 2012, Journal of Neurophysiology.

[37]  Hiroshi Ban,et al.  The integration of motion and disparity cues to depth in dorsal visual cortex , 2012, Nature Neuroscience.

[38]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[39]  Hiroyasu Ujike,et al.  Depth thresholds of motion parallax as a function of head movement velocity , 2001, Vision Research.

[40]  J. Gibson,et al.  Motion parallax as a determinant of perceived depth. , 1959, Journal of experimental psychology.

[41]  S. Rushton,et al.  Optic Flow Processing for the Assessment of Object Movement during Ego Movement , 2009, Current Biology.

[42]  Joseph S. Lappin,et al.  Rogers and Graham's 1979 Paper , 2009 .

[43]  Rufin Vogels,et al.  Convergence of Depth from Texture and Depth from Disparity in Macaque Inferior Temporal Cortex , 2004, The Journal of Neuroscience.

[44]  B. Anderson.  Depth perception , 2008 .

[45]  Mark Nawrot,et al.  Modeling depth from motion parallax with the motion/pursuit ratio , 2014, Front. Psychol..

[46]  Mark Nawrot,et al.  In Pursuit of Perspective: Does Vertical Perspective Disambiguate Depth from Motion Parallax? , 2013, Perception.

[47]  S. Gilson,et al.  Cue combination for 3D location judgements. , 2011, Journal of vision.

[48]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[49]  Gregory C. DeAngelis,et al.  Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons , 2013, Nature Reviews Neuroscience.

[50]  Ying Zhang,et al.  The effect of overall stimulus velocity on motion parallax , 2008, Visual Neuroscience.

[51]  Kenneth H. Britten,et al.  Mechanisms of self-motion perception. , 2008, Annual review of neuroscience.

[52]  Mark Nawrot,et al.  The motion/pursuit law for visual depth perception from motion parallax , 2009, Vision Research.

[53]  M Nawrot,et al.  Neural integration of information specifying structure from stereopsis and motion. , 1989, Science.

[54]  M Lappe,et al.  Dynamical use of different sources of information in heading judgments from retinal flow. , 1999, Journal of the Optical Society of America. A, Optics, image science, and vision.

[55]  A. Parker.  Binocular depth perception and the cerebral cortex , 2007, Nature Reviews Neuroscience.

[56]  K. Hayashibe,et al.  Reversals of Visual Depth Caused by Motion Parallax , 1991, Perception.

[57]  Melvyn A. Goodale,et al.  Distance estimation in the mongolian gerbil: The role of dynamic depth cues , 1984, Behavioural Brain Research.

[58]  G. DeAngelis,et al.  How Can Single Sensory Neurons Predict Behavior? , 2015, Neuron.

[59]  H. Helmholtz.  Helmholtz's Treatise on Physiological Optics , 1963 .

[60]  S. Rushton,et al.  Moving observers, relative retinal motion and the detection of object movement , 2005, Current Biology.

[61]  J H Rieger,et al.  Processing differential image motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[62]  Bruce G Cumming,et al.  Decision-related activity in sensory neurons: correlations among neurons and with behavior. , 2012, Annual review of neuroscience.

[63]  G. DeAngelis,et al.  Fine Discrimination Training Alters the Causal Contribution of Macaque Area MT to Depth Perception , 2008, Neuron.

[64]  Carlos R. Ponce,et al.  Contributions of Indirect Pathways to Visual Response Properties in Macaque Middle Temporal Area MT , 2011, The Journal of Neuroscience.

[65]  B J Rogers,et al.  The Appearance of Surfaces Specified by Motion Parallax and Binocular Disparity , 1989, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[66]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[67]  Mark Nawrot,et al.  Depth from motion parallax scales with eye movement gain. , 2003, Journal of vision.

[68]  W. Newsome,et al.  Estimates of the Contribution of Single Neurons to Perception Depend on Timescale and Noise Correlation , 2009, The Journal of Neuroscience.

[69]  H. Ono,et al.  Depth perception as a function of motion parallax and absolute-distance information. , 1986, Journal of experimental psychology. Human perception and performance.

[70]  HyungGoo R. Kim,et al.  A novel role for visual perspective cues in the neural computation of depth , 2014, Nature Neuroscience.

[71]  Ian P. Howard,et al.  Binocular Vision and Stereopsis , 1996 .

[72]  S. Rushton,et al.  The pop out of scene-relative object movement against retinal motion due to self-movement , 2007, Cognition.

[73]  P. H. Schiller,et al.  Neural responses to relative speed in the primary visual cortex of rhesus monkey , 2003, Visual Neuroscience.

[74]  M. Bethge,et al.  Inferring decoding strategies from choice probabilities in the presence of correlated variability , 2013, Nature Neuroscience.

[75]  G. DeAngelis,et al.  Cortical area MT and the perception of stereoscopic depth , 1998, Nature.

[76]  B J Rogers,et al.  Visual and nonvisual information disambiguate surfaces specified by motion parallax , 1992, Perception & psychophysics.

[77]  C J Duffy,et al.  Optic flow analysis for self-movement perception. , 2000, International review of neurobiology.

[78]  Gregory C DeAngelis,et al.  Coding of horizontal disparity and velocity by MT neurons in the alert macaque. , 2003, Journal of neurophysiology.

[79]  Christopher R Fetsch,et al.  Neural correlates of reliability-based cue weighting during multisensory integration , 2011, Nature Neuroscience.

[80]  H. Sakata,et al.  Integration of perspective and disparity cues in surface-orientation-selective neurons of area CIP. , 2001, Journal of neurophysiology.

[81]  Brian Rogers,et al.  Motion parallax as an independent cue for depth perception: a retrospective. , 2009, Perception.

[82]  Hiroyasu Ujike,et al.  Motion Parallax Driven by Head Movements: Conditions for Visual Stability, Perceived Depth, and Perceived Concomitant Motion , 2005, Perception.