Depth estimation from retinal disparity requires eye and head orientation signals.

To reach for an object, one needs to know its egocentric distance (absolute depth). It remains an unresolved issue which signals are required by the brain to calculate this absolute depth information. We devised a geometric model of binocular 3D eye orientation and investigated the signals necessary to uniquely determine the depth of a non-foveated object accounting for naturalistic variations of eye and head orientations. Our model shows that, in the presence of noisy internal estimates of the ocular vergence angle, horizontal and vertical retinal disparities alone are insufficient to calculate the unique depth of a point-like target. Instead the brain must account for the 3D orientations of the eye and head. We tested the model in a behavioral experiment that involved reaches to targets in depth. Our analysis showed that a target with the same retinal disparity produced different estimates of reach depth that varied consistently with different eye and head orientations. The experimental results showed that subjects accurately account for this extraretinal information when they reach. In summary, when estimating the distance of point-like targets, all available signals about the object's location as well as body configuration are combined to provide accurate information about the object's distance.

[1]  W C Gogel,et al.  Scalar perceptions with binocular cues of distance. , 1972, The American journal of psychology.

[2]  Hiroshi Ono,et al.  The cyclopean eye in vision: the new and old data continue to hit you right between the eyes , 2002, Vision Research.

[3]  K. Hepp On Listing's law , 1990 .

[4]  L. Snyder Coordinate transformations for eye and arm movements in the brain , 2000, Current Opinion in Neurobiology.

[5]  H Collewijn,et al.  Binocular eye movements and the perception of depth. , 1990, Reviews of oculomotor research.

[6]  W Pieter Medendorp,et al.  Updating target distance across eye movements in depth. , 2008, Journal of neurophysiology.

[7]  H. Collewijn,et al.  Visually induced cycloversion and cyclovergence , 1992, Vision Research.

[8]  H. Komatsu,et al.  Disparity sensitivity of neurons in monkey extrastriate area MST , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[9]  T. Bando,et al.  Human Cortical Areas Activated in Relation to Vergence Eye Movements—A PET Study , 1999, NeuroImage.

[10]  Casper J. Erkelens,et al.  A computational model of depth perception based on headcentric disparity , 1998, Vision Research.

[11]  G. Mather The Use of Image Blur as a Depth Cue , 1997, Perception.

[12]  Vieth Ueber die Richtung der Augen , 1818 .

[13]  Peter Guthrie Tait,et al.  An Elementary Treatise on Quaternions , 2010 .

[14]  D. Tweed,et al.  Three-dimensional model of the human eye-head saccadic system. , 1997, Journal of neurophysiology.

[15]  Y Trotter,et al.  Distance Perception within near Visual Space , 2001, Perception.

[16]  Douglas Tweed,et al.  PII: S0042-6989(97)00002-3 , 2003 .

[17]  Tutis Vilis,et al.  Eye position signals modulate early dorsal and ventral visual areas. , 2002, Cerebral cortex.

[18]  James A. Crowell,et al.  Horizontal and vertical disparity, eye position, and stereoscopic slant perception , 1999, Vision Research.

[19]  Ronald S. Harwerth,et al.  Behavioral studies of local stereopsis and disparity vergence in monkeys , 1995, Vision Research.

[20]  Ian P. Howard,et al.  Binocular Vision and Stereopsis , 1996 .

[21]  Francesco Lacquaniti,et al.  Multiple levels of representation of reaching in the parieto-frontal network. , 2003, Cerebral cortex.

[22]  Gregory C DeAngelis,et al.  Binocular Vision: An Orientation to Disparity Coding , 2002, Current Biology.

[23]  E. Johnston Systematic distortions of shape from stereopsis , 1991, Vision Research.

[24]  I. Howard,et al.  Cycloversion and cyclovergence: the effects of the area and position of the visual display , 1994, Experimental Brain Research.

[25]  Hideo Sakata,et al.  Neural mechanisms of three-dimensional vision , 2005, Neuroscience Research.

[27]  M. Banks,et al.  Perceiving slant about a horizontal axis from stereopsis. , 2001, Journal of vision.

[28]  Qing Yang,et al.  Saccade–vergence dynamics and interaction in children and in adults , 2004, Experimental Brain Research.

[29]  Thomas Haslwanter,et al.  Three-dimensional eye position during static roll and pitch in humans , 2001, Vision Research.

[30]  J. E. W. Mayhew,et al.  A computational model of binocular depth perception , 1982, Nature.

[31]  John M. Allman,et al.  The Effect of Gaze Angle and Fixation Distance on the Responses of Neurons in V1, V2, and V4 , 2002, Neuron.

[32]  Kenneth N. Ogle,et al.  INDUCED SIZE EFFECT: I. A NEW PHENOMENON IN BINOCULAR SPACE PERCEPTION ASSOCIATED WITH THE RELATIVE SIZES OF THE IMAGES OF THE TWO EYES , 1938 .

[33]  R. Andersen,et al.  Head position signals used by parietal neurons to encode locations of visual stimuli , 1995, Nature.

[34]  V. Henn,et al.  Static roll and pitch in the monkey: Shift and rotation of listing's plane , 1992, Vision Research.

[35]  P O Bishop,et al.  Vertical disparity, egocentric distance and stereoscopic depth constancy: a new interpretation , 1989, Proceedings of the Royal Society of London. B. Biological Sciences.

[36]  D. Y. P. Henriques,et al.  Direction-dependent distortions of retinocentric space in the visuomotor transformation for pointing , 2000, Experimental Brain Research.

[37]  CASPER J ERKELENS,et al.  Trajectories of the Human Binocular Fixation Point during Conjugate and Non-conjugate Gaze-shifts , 1997, Vision Research.

[38]  Gregory C DeAngelis,et al.  Macaque Middle Temporal Neurons Signal Depth in the Absence of Motion , 2003, The Journal of Neuroscience.

[39]  J. Douglas Crawford,et al.  The motor side of depth vision , 2001, Nature.

[40]  A. Milner,et al.  Perception and Action in Depth , 1998, Consciousness and Cognition.

[41]  Aldo Genovesio,et al.  Integration of retinal disparity and fixation-distance related signals toward an egocentric coding of distance in the posterior parietal cortex of primates. , 2004, Journal of neurophysiology.

[42]  A. Parker,et al.  Comparing perceptual signals of single V5/MT neurons in two binocular depth tasks. , 2004, Journal of neurophysiology.

[43]  C. Schor,et al.  Adaptive Control of Vergence in Humans , 2002, Annals of the New York Academy of Sciences.

[44]  James R. Tresilian,et al.  Increasing confidence in vergence as a cue to distance , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[45]  T Vilis,et al.  Axes of eye rotation and Listing's law during rotations of the head. , 1991, Journal of neurophysiology.

[46]  Gunnar Blohm,et al.  Computations for geometrically accurate visually guided reaching in 3-D space. , 2007, Journal of vision.

[47]  J. D. Crawford,et al.  Spatial Transformations for Eye–Hand Coordination , 2004 .

[48]  A. Parker,et al.  Integration of depth modules: Stereopsis and texture , 1993, Vision Research.

[49]  R Perez,et al.  Neural mechanisms underlying stereoscopic vision , 1998, Progress in Neurobiology.

[50]  M F Bradshaw,et al.  Disparity Scaling and the Perception of Frontoparallel Surfaces , 1995, Perception.

[51]  A. Parker,et al.  Cortical mechanisms of binocular stereoscopic vision. , 2001, Progress in brain research.

[52]  A. V. D. Berg,et al.  Binocular eye orientation during fixations: Listing's law extended to include eye vergence , 1993, Vision Research.

[53]  A. V. van den Berg,et al.  Visually evoked cyclovergence and extended listing's law. , 2000, Journal of neurophysiology.

[54]  R Sekuler,et al.  Blur and Contrast as Pictorial Depth Cues , 1997, Perception.

[55]  M. Mon-Williams,et al.  Some Recent Studies on the Extraretinal Contribution to Distance Perception , 1999, Perception.

[56]  M. Ritter,et al.  Effect of disparity and viewing distance on perceived depth , 1977 .

[57]  James R. Tresilian,et al.  Vergence provides veridical depth perception from horizontal retinal image disparities , 2000, Experimental Brain Research.

[58]  E. Brenner,et al.  Judging distance from ocular convergence , 1998, Vision Research.

[59]  G C DeAngelis,et al.  The physiology of stereopsis. , 2001, Annual review of neuroscience.

[60]  M. Goodale,et al.  An evolving view of duplex vision: separate but interacting cortical pathways for perception and action , 2004, Current Opinion in Neurobiology.

[61]  T. Sejnowski,et al.  A neural model of the cortical representation of egocentric distance. , 1994, Cerebral cortex.

[62]  J. A. Gisbergen,et al.  32 – Conjugate and Disconjugate Contributions to Bifoveal Fixations Studied from a 3D Perspective , 1994 .

[63]  Whitman Richards,et al.  Convergence as a cue to depth , 1969 .

[64]  T. Haslwanter Mathematics of three-dimensional eye rotations , 1995, Vision Research.

[65]  G. DeAngelis,et al.  Cortical area MT and the perception of stereoscopic depth , 1998, Nature.

[66]  L. Brand,et al.  Vector and tensor analysis , 1947 .

[67]  D. Tweed,et al.  Stereopsis Outweighs Gravity in the Control of the Eyes , 2001, The Journal of Neuroscience.

[68]  T. Vilis,et al.  Rotation of Listing's plane during vergence , 1992, Vision Research.

[69]  Peter Neri,et al.  A stereoscopic look at visual cortex. , 2005, Journal of neurophysiology.

[70]  M A Goodale,et al.  Different spaces and different times for perception and action. , 2001, Progress in brain research.

[71]  Hiroshi Ono,et al.  The cyclopean eye is relevant for predicting visual direction , 2005, Vision Research.

[72]  J. M. Foley Binocular distance perception. , 1980, Psychological review.

[73]  Kai M Schreiber,et al.  The extended horopter: quantifying retinal correspondence across changes of 3D eye position. , 2006, Journal of vision.

[74]  Izumi Ohzawa,et al.  Mechanisms of stereoscopic vision: the disparity energy model , 1998, Current Opinion in Neurobiology.

[75]  Berthold K. P. Horn Relative orientation , 1987, International Journal of Computer Vision.

[76]  Ian S. Curthoys,et al.  On the relation between ocular torsion and visual perception of line orientation , 2008, Vision Research.

[77]  J R Tresilian,et al.  Ordinal depth information from accommodation? , 2000, Ergonomics.

[78]  R. Andersen,et al.  Visual receptive field organization and cortico‐cortical connections of the lateral intraparietal area (area LIP) in the macaque , 1990, The Journal of comparative neurology.

[79]  George Sperling,et al.  A gain-control theory of binocular combination. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[80]  Raymond van Ee,et al.  The influence of cyclovergence on unconstrained stereoscopic matching , 2003, Vision Research.

[81]  L E Mays,et al.  Neurons in monkey parietal area LIP are tuned for eye-movement parameters in three-dimensional space. , 1995, Journal of neurophysiology.

[82]  S. Blackburn,et al.  Contrast as a depth cue , 1994, Vision Research.

[83]  William Rowan Hamilton,et al.  Elements of Quaternions , 1969 .

[84]  David C Knill,et al.  Reaching for visual cues to depth: the brain combines depth cues differently for motor control and perception. , 2005, Journal of vision.

[85]  A P Batista,et al.  Reach plans in eye-centered coordinates. , 1999, Science.

[86]  Nikos A. Aspragathos,et al.  A comparative study of three methods for robot kinematics , 1998, IEEE Trans. Syst. Man Cybern. Part B.