Optimal Multimodal Integration in Spatial Localization

Saccadic eye movements facilitate rapid and efficient exploration of visual scenes, but also pose serious challenges to establishing reliable spatial representations. This process presumably depends on extraretinal information about eye position, but it is still unclear whether afferent or efferent signals are implicated and how these signals are combined with the visual input. Using a novel gaze-contingent search paradigm with highly controlled retinal stimulation, we examined the performance of human observers in locating a previously fixated target after a variable number of saccades, a task that generates contrasting predictions for different updating mechanisms. We show that while localization accuracy is unaffected by saccades, localization precision deteriorates nonlinearly, revealing a statistically optimal combination of retinal and extraretinal signals. These results provide direct evidence for optimal multimodal integration in the updating of spatial representations and elucidate the contributions of corollary discharge signals and eye proprioception.

[1]  Robert M. McPeek,et al.  Concurrent processing of saccades in visual search , 2000, Vision Research.

[2]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[3]  W. Geisler,et al.  Contributions of ideal observer theory to vision research , 2011, Vision Research.

[4]  F. Hamker,et al.  A Computational Model for the Influence of Corollary Discharge and Proprioception on the Perisaccadic Mislocalization of Briefly Presented Stimuli in Complete Darkness , 2011, The Journal of Neuroscience.

[5]  C. Sherrington.  OBSERVATIONS ON THE SENSUAL RÔLE OF THE PROPRIOCEPTIVE NERVE-SUPPLY OF THE EXTRINSIC OCULAR MUSCLES , 1918 .

[6]  Daniela Balslev,et al.  The University of Birmingham (live System) Eye Position Representation in Human Anterior Parietal Cortex Eye Position Representation in Human Anterior Parietal Cortex , 2022 .

[7]  Paul A. Warren,et al.  A Bayesian Model of Perceived Head-Centered Velocity during Smooth Pursuit Eye Movement , 2010, Current Biology.

[8]  A. A. Skavenski,et al.  Contr of eye position in the dark. , 1970, Vision research.

[9]  J. Douglas Crawford,et al.  Optimal transsaccadic integration explains distorted spatial perception , 2003, Nature.

[10]  I. Donaldson,et al.  The functions of the proprioceptors of the eye muscles. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[11]  P. E. Hallett,et al.  Saccadic eye movements towards stimuli triggered by prior saccades , 1976, Vision Research.

[12]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[13]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[14]  Eero P. Simoncelli,et al.  Noise characteristics and prior expectations in human visual speed perception , 2006, Nature Neuroscience.

[15]  W Pieter Medendorp,et al.  Spatial constancy mechanisms in motor control , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[16]  R. Sperry.  Neural basis of the spontaneous optokinetic response produced by visual inversion. , 1950, Journal of comparative and physiological psychology.

[17]  Jean-René Duhamel,et al.  Optimal Sensorimotor Control in Eye Movement Sequences , 2009, The Journal of Neuroscience.

[18]  Christopher J. Peck,et al.  The time course of the tonic oculomotor proprioceptive signal in area 3a of somatosensory cortex. , 2011, Journal of neurophysiology.

[19]  D. Zee,et al.  Oculomotor function in the rhesus monkey after deafferentation of the extraocular muscles , 2001, Experimental Brain Research.

[20]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[21]  A. A. Skavenski.  Inflow as a source of extraretinal eye position information. , 1972, Vision research.

[22]  W Li,et al.  Visual Direction Is Corrected by a Hybrid Extraretinal Eye Position Signal a , 1992, Annals of the New York Academy of Sciences.

[23]  Christopher R Fetsch,et al.  Neural correlates of reliability-based cue weighting during multisensory integration , 2011, Nature Neuroscience.

[24]  Michele Rucci,et al.  EyeRIS: A general-purpose system for eye-movement-contingent display control , 2007, Behavior research methods.

[25]  M. Hayhoe,et al.  Reference frames in saccadic targeting , 1997, Experimental Brain Research.

[26]  R. Wurtz.  Neuronal mechanisms of visual stability , 2008, Vision Research.

[27]  K. F. Austen,et al.  Oculoparalytic Illusion : Visual-Field Dependent Spatial Mislocalizations by Humans Partially Paralyzed with Curare , 2006 .

[28]  R. Steinman.  Effect of Target Size, Luminance, and Color on Monocular Fixation* , 1965 .

[29]  H. Akaike.  A new look at the statistical model identification , 1974 .

[30]  J. Velay,et al.  Shift in saccadic direction induced in humans by proprioceptive manipulation: a comparison between memory-guided and visually guided saccades , 1996, Experimental Brain Research.

[31]  Thérèse Collins,et al.  Extraretinal signal metrics in multiple-saccade sequences. , 2010, Journal of vision.

[32]  Michael F Land,et al.  Vision and the representation of the surroundings in spatial memory , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[33]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[34]  J. Vercher,et al.  The role of ocular muscle proprioception in visual localization of targets. , 1990, Science.

[35]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[36]  M. Steinbach,et al.  On the role of extraretinal signals for saccade generation , 1995, Experimental Brain Research.

[37]  D. Sparks,et al.  Corollary discharge provides accurate eye position information to the oculomotor system. , 1983, Science.

[38]  G. Lennerstrand,et al.  Effects of eye muscle proprioceptive activation on eye position in normal and exotropic subjects , 1997, Graefe's Archive for Clinical and Experimental Ophthalmology.

[39]  Arnold Ziesche,et al.  Computational models of spatial updating in peri-saccadic perception , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[40]  K. Ren,et al.  Regulation of spinal neuropeptide genes in a rat model of peripheral inflammation and hyperalgesia. , 1995, Progress in brain research.

[41]  Martina Poletti,et al.  Stability of the Visual World during Eye Drift , 2010, The Journal of Neuroscience.

[42]  L. Stark,et al.  Ocular proprioception and efference copy in registering visual direction , 1991, Vision Research.

[43]  Wei Ji Ma,et al.  Spiking networks for Bayesian inference and choice , 2008, Current Opinion in Neurobiology.

[44]  R. Wurtz,et al.  A Pathway in Primate Brain for Internal Monitoring of Movements , 2002, Science.