Binocular Mechanisms of 3D Motion Processing.

The visual system must recover important properties of the external environment if its host is to survive. Because the retinae are effectively two-dimensional but the world is three-dimensional (3D), the patterns of stimulation both within and across the eyes must be used to infer the distal stimulus-the environment-in all three dimensions. Moreover, animals and elements in the environment move, which means the input contains rich temporal information. Here, in addition to reviewing the literature, we discuss how and why prior work has focused on purported isolated systems (e.g., stereopsis) or cues (e.g., horizontal disparity) that do not necessarily map elegantly on to the computations and complex patterns of stimulation that arise when visual systems operate within the real world. We thus also introduce the binoptic flow field (BFF) as a description of the 3D motion information available in realistic environments, which can foster the use of ecologically valid yet well-controlled stimuli. Further, it can help clarify how future studies can more directly focus on the computations and stimulus properties the visual system might use to support perception and behavior in a dynamic 3D world.

[1]  Charles Wheatstone On some remarkable and hitherto unobserved phenomena of binocular vision. , 1962 .

[2]  C Blakemore,et al.  On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images , 1969, The Journal of physiology.

[3]  F. Bremmer,et al.  Spatial invariance of visual receptive fields in parietal cortex neurons , 1997, Nature.

[4]  Martin Lages,et al.  On the Inverse Problem of Binocular 3D Motion Perception , 2010, PLoS Comput. Biol..

[5]  Jenny C A Read,et al.  Stereo correspondence is optimized for large viewing distances , 2010, The European journal of neuroscience.

[6]  D. Regan,et al.  Binocular and monocular stimuli for motion in depth: Changing-disparity and changing-size feed the same motion-in-depth stage , 1979, Vision Research.

[7]  Yukio Komatsu,et al.  Responsiveness of Clare-Bishop neurons to visual cues associated with motion of a visual stimulus in three-dimensional space , 1985, Vision Research.

[8]  Gregory C DeAngelis,et al.  Coding of horizontal disparity and velocity by MT neurons in the alert macaque. , 2003, Journal of neurophysiology.

[9]  C. Quaia,et al.  Human short-latency ocular vergence responses produced by interocular velocity differences , 2016, Journal of vision.

[10]  Thaddeus B. Czuba,et al.  Three-dimensional motion aftereffects reveal distinct direction-selective mechanisms for binocular processing of motion through depth. , 2011, Journal of vision.

[11]  B G Cumming The Relationship between Stereoacuity and Stereomotion Thresholds , 1995, Perception.

[12]  Bertram E. Shi,et al.  Neural population models for perception of motion in depth , 2014, Vision Research.

[13]  K. Hoffmann,et al.  Linear Vestibular Self‐Motion Signals in Monkey Medial Superior Temporal Area , 1999, Annals of the New York Academy of Sciences.

[14]  Julie M. Harris,et al.  Poor visibility of motion in depth is due to early motion averaging , 2003, Vision Research.

[15]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[16]  D Regan,et al.  Visual field defects for vergence eye movements and for stereomotion perception. , 1986, Investigative ophthalmology & visual science.

[17]  G. DeAngelis,et al.  Cortical area MT and the perception of stereoscopic depth , 1998, Nature.

[18]  Rob Gray,et al.  Binocular processing of motion: some unresolved questions. , 2009, Spatial vision.

[19]  D. Regan,et al.  Neurons in cat visual cortex tuned to the direction of motion in depth: Effect of positional disparity , 1982, Vision Research.

[20]  M Lappe,et al.  Computational mechanisms for optic flow analysis in primate cortex. , 2000, International review of neurobiology.

[21]  C. Duffy MST neurons respond to optic flow and translational movement. , 1998, Journal of neurophysiology.

[22]  D. Regan,et al.  Visual field defects for unidirectional and oscillatory motion in depth , 1989, Vision Research.

[23]  C. Wheatstone XVIII. Contributions to the physiology of vision. —Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision , 1962, Philosophical Transactions of the Royal Society of London.

[24]  O. Braddick A short-range process in apparent motion. , 1974, Vision research.

[25]  S. Sterbing-D’Angelo,et al.  Behavioral/systems/cognitive Multisensory Space Representations in the Macaque Ventral Intraparietal Area , 2022 .

[26]  K. H. Britten,et al.  Responses of neurons in macaque MT to stochastic motion signals , 1993, Visual Neuroscience.

[27]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[28]  Wyeth Bair,et al.  A Model of Binocular Motion Integration in MT Neurons , 2016, The Journal of Neuroscience.

[29]  James A. Crowell,et al.  Horizontal and vertical disparity, eye position, and stereoscopic slant perception , 1999, Vision Research.

[30]  D. Regan,et al.  Cyclopean Discrimination Thresholds for the Direction and Speed of Motion in Depth , 1996, Vision Research.

[31]  Frank Bremmer,et al.  ã Federation of European Neuroscience Societies Heading encoding in the macaque ventral intraparietal area (VIP) , 2022 .

[32]  Lora T. Likova,et al.  Stereomotion processing in the human occipital cortex , 2007, NeuroImage.

[33]  B. C. Motter,et al.  Responses of neurons in visual cortex (V1 and V2) of the alert macaque to dynamic random-dot stereograms , 1985, Vision Research.

[34]  Kazumichi Matsumiya,et al.  Motion mechanisms with different spatiotemporal characteristics identified by an MAE technique with superimposed gratings. , 2009, Journal of vision.

[35]  D. Nilsson Eyes as optical alarm systems in fan worms and ark clams , 1994 .

[36]  Michael F. Land,et al.  From eye movements to actions: how batsmen hit the ball , 2000, Nature Neuroscience.

[37]  John H. R. Maunsell,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. , 1983, Journal of neurophysiology.

[38]  John H. R. Maunsell,et al.  The middle temporal visual area in the macaque: Myeloarchitecture, connections, functional properties and topographic organization , 1981, The Journal of comparative neurology.

[39]  M. J. Morgan,et al.  Conditions for motion flow in dynamic visual noise , 1980, Vision Research.

[40]  S. Dumoulin,et al.  Impaired Velocity Processing Reveals an Agnosia for Motion in Depth , 2016, Psychological science.

[41]  Thaddeus B. Czuba,et al.  Separate Perceptual and Neural Processing of Velocity- and Disparity-Based 3D Motion Signals , 2016, The Journal of Neuroscience.

[42]  Leland S Stone,et al.  Stereomotion speed perception: contributions from both changing disparity and interocular velocity difference over a range of relative disparities. , 2004, Journal of vision.

[43]  H. K. Nishihara,et al.  Practical Real-Time Imaging Stereo Matcher , 1984 .

[44]  Yong Gu,et al.  Evidence for a Causal Contribution of Macaque Vestibular, But Not Intraparietal, Cortex to Heading Perception , 2016, The Journal of Neuroscience.

[45]  C Shawn Green,et al.  Optimal Combination of the Binocular Cues to 3D Motion. , 2015, Investigative ophthalmology & visual science.

[46]  Dora E Angelaki,et al.  Eye-centered visual receptive fields in the ventral intraparietal area. , 2014, Journal of neurophysiology.

[47]  Michael N. Shadlen,et al.  Dichoptic activation of the early motion system , 1993, Vision Research.

[48]  Leland S Stone,et al.  Stereomotion suppression and the perception of speed: accuracy and precision as a function of 3D trajectory. , 2006, Journal of vision.

[49]  G. Orban,et al.  Responses of macaque STS neurons to optic flow components: a comparison of areas MT and MST. , 1994, Journal of neurophysiology.

[50]  Patrick Cavanagh,et al.  The reference frame of the motion aftereffect is retinotopic. , 2009, Journal of vision.

[51]  Gregory C. DeAngelis,et al.  A neural representation of depth from motion parallax in macaque visual cortex , 2008, Nature.

[52]  R. Wurtz,et al.  The role of disparity-sensitive cortical neurons in signalling the direction of self-motion , 1990, Nature.

[53]  M. Cynader,et al.  Neurones in cat parastriate cortex sensitive to the direction of motion in three‐dimensional space , 1978, The Journal of physiology.

[54]  Thomas D. Albright,et al.  The Complex Structure of Receptive Fields in the Middle Temporal Area , 2013, Front. Syst. Neurosci..

[55]  L. Cormack,et al.  Disparity- and velocity-based signals for three-dimensional motion perception in human MT+ , 2009, Nature Neuroscience.

[56]  Julie M. Harris,et al.  What Visual Information is Used for Stereoscopic Depth Displacement Discrimination? , 2010, Perception.

[57]  S. Zeki Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey , 1974, The Journal of physiology.

[58]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[59]  B. G. Cumming,et al.  Binocular mechanisms for detecting motion-in-depth , 1994, Vision Research.

[60]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[61]  I. Ohzawa,et al.  Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. , 1990, Science.

[62]  S. Zeki Cells responding to changing image size and disparity in the cortex of the rhesus monkey , 1974, The Journal of physiology.

[63]  Robert Sekuler,et al.  Coherent global motion percepts from stochastic local motions , 1984, Vision Research.

[64]  Lawrence K. Cormack,et al.  Global eye-specific motion signal for three-dimensional motion processing revealed through adaptation , 2016 .

[65]  Bart Farell,et al.  Motion in depth from interocular velocity differences revealed by differential motion aftereffect , 2006, Vision Research.

[66]  S. Dumoulin,et al.  Transformation from a Retinal to a Cyclopean Representation in Human Visual Cortex , 2015, Current Biology.

[67]  K. Toyama,et al.  The responsiveness of Clare-Bishop neurons to motion cues for motion stereopsis , 1986, Neuroscience Research.

[68]  Anthony J. Movshon,et al.  Visual Response Properties of Striate Cortical Neurons Projecting to Area MT in Macaque Monkeys , 1996, The Journal of Neuroscience.

[69]  Thaddeus B. Czuba,et al.  Area MT Encodes Three-Dimensional Motion , 2014, The Journal of Neuroscience.

[70]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[71]  G. Poggio,et al.  Mechanisms of static and dynamic stereopsis in foveal cortex of the rhesus monkey , 1981, The Journal of physiology.

[72]  D. Mitchell,et al.  Interocular transfer of the motion after-effect in normal and stereoblind observers , 2004, Experimental Brain Research.

[73]  Thaddeus B. Czuba,et al.  Speed and eccentricity tuning reveal a central role for the velocity-based cue to 3D visual motion. , 2010, Journal of neurophysiology.

[74]  R. Born,et al.  Integrating motion and depth via parallel pathways , 2008, Nature Neuroscience.

[75]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[76]  Frank Bremmer,et al.  Interaction of linear vestibular and visual stimulation in the macaque ventral intraparietal area (VIP) , 2002, The European journal of neuroscience.

[77]  Bart Farell,et al.  Seeing motion in depth using inter-ocular velocity differences , 2005, Vision Research.

[78]  Kenneth H Britten,et al.  Area MST and heading perception in macaque monkeys. , 2002, Cerebral cortex.

[79]  Thaddeus B. Czuba,et al.  To CD or not to CD: Is there a 3D motion aftereffect based on changing disparities? , 2012, Journal of vision.

[80]  G. DeAngelis,et al.  A Logarithmic, Scale-Invariant Representation of Speed in Macaque Middle Temporal Area Accounts for Speed Discrimination Performance , 2005, The Journal of Neuroscience.

[81]  Bertram E. Shi,et al.  The changing disparity energy model , 2010, Vision Research.

[82]  John H. R. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[83]  Kenneth H. Britten,et al.  Mechanisms of self-motion perception. , 2008, Annual review of neuroscience.

[84]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[85]  C. Blakemore The range and scope of binocular depth discrimination in man , 1970, The Journal of physiology.

[86]  J Duysens,et al.  Neurons in the ventral intraparietal area of awake macaque monkey closely resemble neurons in the dorsal part of the medial superior temporal area in their responses to optic flow patterns. , 1996, Journal of neurophysiology.

[87]  R. Wurtz,et al.  Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. , 1991, Journal of neurophysiology.

[88]  M F Bradshaw,et al.  Vertical disparities, differential perspective and binocular stereopsis , 1993, Nature.

[89]  Lawrence K. Cormack,et al.  Reflexive and voluntary control of smooth eye movements , 2013, Electronic Imaging.

[90]  Kevin R Brooks,et al.  Monocular motion adaptation affects the perceived trajectory of stereomotion. , 2002, Journal of experimental psychology. Human perception and performance.

[91]  D. Regan,et al.  Binocular correlates of the direction of motion in depth , 1993, Vision Research.

[92]  S. Dumoulin,et al.  Stereomotion scotomas occur after binocular combination , 2014, Vision Research.

[93]  Yong Gu,et al.  Causal Links between Dorsal Medial Superior Temporal Area Neurons and Multisensory Heading Perception , 2012, The Journal of Neuroscience.

[94]  T. Albright Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[95]  A. V. van den Berg,et al.  Why two eyes are better than one for judgements of heading , 1994, Nature.

[96]  R. M. Siegel,et al.  Maps of Visual Space in Human Occipital Cortex Are Retinotopic, Not Spatiotopic , 2008, The Journal of Neuroscience.

[97]  Thaddeus B. Czuba,et al.  Motion processing with two eyes in three dimensions. , 2011, Journal of vision.

[98]  Gregory C DeAngelis,et al.  Neural Representation of Motion-In-Depth in Area MT , 2014, The Journal of Neuroscience.

[99]  E. Adelson,et al.  Phenomenal coherence of moving visual patterns , 1982, Nature.

[100]  Kevin R Brooks,et al.  Interocular velocity difference contributes to stereomotion speed perception. , 2002, Journal of vision.

[101]  Bas Rokers,et al.  Strong percepts of motion through depth without strong percepts of position in depth. , 2008, Journal of vision.

[102]  A. Norcia,et al.  Temporal frequency limits for stereoscopic apparent motion processes , 1984, Vision Research.

[103]  S. Anstis,et al.  Phi movement as a subtraction process. , 1970, Vision research.

[104]  Tao Zhang,et al.  Parietal Area VIP Causally Influences Heading Perception during Pursuit Eye Movements , 2011, The Journal of Neuroscience.