Multisensory space: from eye‐movements to self‐motion

We perceive the world around us as stable. This is remarkable given that our body parts as well as we ourselves are constantly in motion. Humans and other primates move their eyes more often than their hearts beat. Such eye movements lead to coherent motion of the images of the outside world across the retina. Furthermore, during everyday life, we constantly approach targets, avoid obstacles or otherwise move in space. These movements induce motion across different sensory receptor epithels: optical flow across the retina, tactile flow across the body surface and even auditory flow as detected from the two ears. It is generally assumed that motion signals as induced by one's own movement have to be identified and differentiated from the real motion in the outside world. In a number of experimental studies we and others have functionally characterized the primate posterior parietal cortex (PPC) and its role in multisensory encoding of spatial and motion information. Extracellular recordings in the macaque monkey showed that during steady fixation the visual, auditory and tactile spatial representations in the ventral intraparietal area (VIP) are congruent. This finding was of major importance given that a functional MRI (fMRI) study determined the functional equivalent of macaque area VIP in humans. Further recordings in other areas of the dorsal stream of the visual cortical system of the macaque pointed towards the neural basis of perceptual phenomena (heading detection during eye movements, saccadic suppression, mislocalization of visual stimuli during eye movements) as determined in psychophysical studies in humans.

[1]  Frank Bremmer,et al.  Interaction of linear vestibular and visual stimulation in the macaque ventral intraparietal area (VIP) , 2002, The European journal of neuroscience.

[2]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[3]  S. Sterbing-D’Angelo,et al.  Behavioral/systems/cognitive Multisensory Space Representations in the Macaque Ventral Intraparietal Area , 2022 .

[4]  John H. R. Maunsell,et al.  Functional visual streams , 1992, Current Opinion in Neurobiology.

[5]  Frank Bremmer,et al.  Task influences on the dynamic properties of fast eye movements. , 2009, Journal of vision.

[6]  K. Hoffmann,et al.  Neural Dynamics of Saccadic Suppression , 2009, Journal of Neuroscience.

[7]  K. Hoffmann,et al.  Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. , 1997, Journal of neurophysiology.

[8]  Richard A. Andersen,et al.  Coordinate transformations in the representation of spatial information , 1993, Current Opinion in Neurobiology.

[9]  F. Bremmer,et al.  Localization of visual targets during optokinetic eye movements , 2007, Vision Research.

[10]  Aya Takemura,et al.  MST neurons code for visual motion in space independent of pursuit eye movements. , 2007, Journal of neurophysiology.

[11]  F. Bremmer,et al.  Localization of visual and auditory stimuli during smooth pursuit eye movements. , 2010, Journal of vision.

[12]  G. DeAngelis,et al.  Multimodal Coding of Three-Dimensional Rotation and Translation in Area MSTd: Comparison of Visual and Vestibular Selectivity , 2007, The Journal of Neuroscience.

[13]  K. Hoffmann,et al.  Ocular responses to radial optic flow and single accelerated targets in humans , 1999, Vision Research.

[14]  D. Boussaoud,et al.  Gaze effects in the cerebral cortex: reference frames for space coding and action , 1999, Experimental Brain Research.

[15]  Wolfgang Nitsche,et al.  Infrared based visualization of wall shear stress distributions with a high temporal and spatial resolution , 2007, J. Vis..

[16]  I Daum,et al.  The role of the human thalamus in processing corollary discharge. , 2005, Brain : a journal of neurology.

[17]  K. Hoffmann,et al.  Optokinetic eye movements elicited by radial optic flow in the macaque monkey. , 1998, Journal of neurophysiology.

[18]  Frank Bremmer,et al.  ã Federation of European Neuroscience Societies Heading encoding in the macaque ventral intraparietal area (VIP) , 2022 .

[19]  Guillaume S. Masson,et al.  Motion perception during saccadic eye movements , 2000, Nature Neuroscience.

[20]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[21]  Bremmer,et al.  Eye position encoding in the macaque posterior parietal cortex , 1998, The European journal of neuroscience.

[22]  Marc A Sommer,et al.  Identifying corollary discharges for movement in the primate brain. , 2004, Progress in brain research.

[23]  R. Wurtz,et al.  A Pathway in Primate Brain for Internal Monitoring of Movements , 2002, Science.

[24]  F. Bremmer,et al.  Seeing and Acting at the Same Time Challenges for Brain (and) Research , 2003, Neuron.

[25]  E. Holst,et al.  Das Reafferenzprinzip , 2004, Naturwissenschaften.

[26]  K. Hoffmann,et al.  Selectivity of macaque ventral intraparietal area (area VIP) for smooth pursuit eye movements , 2003, The Journal of physiology.

[27]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[28]  G H Recanzone,et al.  Effects of attention on MT and MST neuronal activity during pursuit initiation. , 2000, Journal of neurophysiology.

[29]  D. Burr,et al.  The effect of optokinetic nystagmus on the perceived position of briefly flashed targets , 2007, Vision Research.

[30]  H. Komatsu,et al.  Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. , 1988, Journal of neurophysiology.

[31]  F. Bremmer,et al.  Localisation of Auditory Targets during Optokinetic Nystagmus , 2007, Perception.

[32]  K. Hoffmann,et al.  Linear Vestibular Self‐Motion Signals in Monkey Medial Superior Temporal Area , 1999, Annals of the New York Academy of Sciences.

[33]  F. Bremmer,et al.  Expansion of visual space during optokinetic afternystagmus (OKAN). , 2008, Journal of neurophysiology.

[34]  R. Sperry Neural basis of the spontaneous optokinetic response produced by visual inversion. , 1950, Journal of comparative and physiological psychology.

[35]  G. DeAngelis,et al.  A functional link between area MSTd and heading perception based on vestibular signals , 2007, Nature Neuroscience.

[36]  M. Morrone,et al.  Extraretinal Control of Saccadic Suppression , 2000, The Journal of Neuroscience.

[37]  C. Duffy MST neurons respond to optic flow and translational movement. , 1998, Journal of neurophysiology.

[38]  K. Hoffmann,et al.  Motion perception during saccades , 1993, Vision Research.

[39]  P. Thier,et al.  Posterior Parietal Cortex Neurons Encode Target Motion in World-Centered Coordinates , 2004, Neuron.

[40]  D M Wolpert,et al.  Sensorimotor integration compensates for visual localization errors during smooth pursuit eye movements. , 2001, Journal of neurophysiology.

[41]  F. Bremmer,et al.  An fMRI study of optokinetic nystagmus and smooth-pursuit eye movements in humans , 2005, Experimental Brain Research.

[42]  F. Bremmer,et al.  Perisaccadic localization of auditory stimuli , 2009, Experimental Brain Research.

[43]  Leslie G. Ungerleider Two cortical visual systems , 1982 .

[44]  David Burr,et al.  Suppression of the magnocellular pathway during saccades , 1996, Behavioural Brain Research.

[45]  U. Ilg Visual-tracking neurons in area MST are activated during anticipatory pursuit eye movements , 2003, Neuroreport.

[46]  H. Komatsu,et al.  Relation of cortical areas MT and MST to pursuit eye movements. III. Interaction with full-field visual stimulation. , 1988, Journal of neurophysiology.

[47]  F Bremmer,et al.  Eye position encoding in the macaque ventral intraparietal area (VIP). , 1999, Neuroreport.

[48]  Mingsha Zhang,et al.  The proprioceptive representation of eye position in monkey primary somatosensory cortex , 2007, Nature Neuroscience.

[49]  R. Wurtz,et al.  Brain circuits for the internal monitoring of movements. , 2008, Annual review of neuroscience.

[50]  François Klam,et al.  ã Federation of European Neuroscience Societies Visual±vestibular interactive responses in the macaque ventral intraparietal area (VIP) , 2022 .

[51]  P. Thier,et al.  A neuronal correlate of spatial stability during periods of self-induced visual motion , 2004, Experimental Brain Research.

[52]  U. Ilg Slow eye movements , 1997, Progress in Neurobiology.

[53]  H. Komatsu,et al.  Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs. , 1988, Journal of neurophysiology.

[54]  Markus Lappe,et al.  Visual selectivity for heading in monkey area MST , 2009, Experimental Brain Research.

[55]  C L Colby,et al.  The analysis of visual space by the lateral intraparietal area of the monkey: the role of extraretinal signals. , 1993, Progress in brain research.

[56]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.