Integration of visual and tactile information in reproduction of traveled distance.

In the natural world, self-motion always stimulates several different sensory modalities. Here we investigated the interplay between a visual optic flow stimulus simulating self-motion and a tactile stimulus (air flow resulting from self-motion) while human observers were engaged in a distance reproduction task. We found that adding congruent tactile information (i.e., speed of the air flow and speed of visual motion are directly proportional) to the visual information significantly improves the precision of the actively reproduced distances. This improvement, however, was smaller than predicted for an optimal integration of visual and tactile information. In contrast, incongruent tactile information (i.e., speed of the air flow and speed of visual motion are inversely proportional) did not improve subjects' precision indicating that incongruent tactile information and visual information were not integrated. One possible interpretation of the results is a link to properties of neurons in the ventral intraparietal area that have been shown to have spatially and action-congruent receptive fields for visual and tactile stimuli.NEW & NOTEWORTHY This study shows that tactile and visual information can be integrated to improve the estimates of the parameters of self-motion. This, however, happens only if the two sources of information are congruent-as they are in a natural environment. In contrast, an incongruent tactile stimulus is still used as a source of information about self-motion but it is not integrated with visual information.

[1]  S. Goldstone,et al.  Studies of Auditory-Visual Differences in Human Time Judgment: 1. Sounds are Judged Longer than Lights , 1974, Perceptual and motor skills.

[2]  J. Dichgans,et al.  Differential effects of central versus peripheral vision on egocentric and exocentric motion perception , 1973, Experimental Brain Research.

[3]  Bill Kapralos,et al.  Auditory Cues in the Perception of Self Motion , 2004 .

[4]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.

[5]  C. Duffy.  MST neurons respond to optic flow and translational movement. , 1998, Journal of neurophysiology.

[6]  Felix Blankenburg,et al.  Tactile and visual motion direction processing in hMT+/V5 , 2014, NeuroImage.

[7]  Suliann Ben Hamed,et al.  Impact Prediction by Looming Visual Stimuli Enhances Tactile Detection , 2015, The Journal of Neuroscience.

[8]  E. Knudsen,et al.  Creating a unified representation of visual and auditory space in the brain. , 1995, Annual review of neuroscience.

[9]  Daniel Västfjäll,et al.  Sound Representing Self-Motion in Virtual Environments Enhances Linear Vection , 2008, PRESENCE: Teleoperators and Virtual Environments.

[10]  Frank Bremmer,et al.  Interaction of linear vestibular and visual stimulation in the macaque ventral intraparietal area (VIP) , 2002, The European journal of neuroscience.

[11]  F. Colavita.  Human sensory dominance , 1974 .

[12]  F. Bremmer,et al.  The use of optical velocities for distance discrimination and reproduction during visually simulated self motion , 1999, Experimental Brain Research.

[13]  A. Serino,et al.  Dynamic Sounds Capture the Boundaries of Peripersonal Space Representation in Humans , 2012, PloS one.

[14]  C. Spence,et al.  Cross-modal links in spatial attention. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[15]  K. Hoffmann,et al.  Optic Flow Processing in Monkey STS: A Theoretical and Experimental Approach , 1996, The Journal of Neuroscience.

[16]  James C Craig,et al.  Visual Motion Interferes with Tactile Motion Perception , 2006, Perception.

[17]  Dora E Angelaki,et al.  Visual and Nonvisual Contributions to Three-Dimensional Heading Selectivity in the Medial Superior Temporal Area , 2006, The Journal of Neuroscience.

[18]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[19]  R. Andersen,et al.  Responses to auditory stimuli in macaque lateral intraparietal area. I. Effects of training. , 1999, Journal of neurophysiology.

[20]  J. Duhamel,et al.  Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey , 2007, The Journal of Neuroscience.

[21]  Jennifer L. Campos,et al.  Bayesian integration of visual and vestibular signals for heading. , 2009, Journal of vision.

[22]  D. Senkowski,et al.  The multifaceted interplay between attention and multisensory integration , 2010, Trends in Cognitive Sciences.

[23]  Suliann Ben Hamed,et al.  Multimodal Convergence within the Intraparietal Sulcus of the Macaque Monkey , 2013, The Journal of Neuroscience.

[24]  Ikuya Murakami,et al.  Perceived duration of visual motion increases with speed. , 2009, Journal of vision.

[25]  R. Hetherington.  The Perception of the Visual World , 1952 .

[26]  J. Wearden,et al.  Why “Sounds Are Judged Longer than Lights”: Application of a Model of the Internal Clock in Humans , 1998, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[27]  M. Bach,et al.  Ambiguity in Tactile Apparent Motion Perception , 2016, PloS one.

[28]  W. Warren,et al.  Perception of translational heading from optical flow. , 1988, Journal of experimental psychology. Human perception and performance.

[29]  Ingo Fründ,et al.  Inference for psychometric functions in the presence of nonstationary behavior. , 2011, Journal of vision.

[30]  François Klam,et al.  ã Federation of European Neuroscience Societies Visual±vestibular interactive responses in the macaque ventral intraparietal area (VIP) , 2022 .

[31]  H. C. Dijkerman,et al.  Visuo-tactile interactions are dependent on the predictive value of the visual stimulus , 2015, Neuropsychologia.

[32]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[33]  Michael S. Beauchamp,et al.  Re-examining overlap between tactile and visual motion responses within hMT+ and STS , 2015, NeuroImage.

[34]  J. T. Walker,et al.  Auditory-visual conflicts in the perceived duration of lights, tones and gaps. , 1981, Journal of experimental psychology. Human perception and performance.

[35]  A. Pouget,et al.  Reference frames for representing visual and tactile locations in parietal cortex , 2005, Nature Neuroscience.

[36]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[37]  K. Tanaka,et al.  Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[38]  Kathleen E Cullen,et al.  The neural encoding of self-motion , 2011, Current Opinion in Neurobiology.

[39]  James B. Maciokas,et al.  Extrastriate Area Mst and Parietal Area Vip Similarly Represent Forward Headings Subjects and Surgery , 2022 .

[40]  T. Mergner,et al.  Interaction of vestibular and proprioceptive inputs for human self-motion perception , 1992, Neuroscience Letters.

[41]  Frank Bremmer,et al.  Self-motion reproduction can be affected by associated auditory cues. , 2011, Seeing and perceiving.

[42]  Richard Granger,et al.  A cortical model of winner-take-all competition via lateral inhibition , 1992, Neural Networks.

[43]  Christopher R Fetsch,et al.  Dynamic Reweighting of Visual and Vestibular Cues during Self-Motion Perception , 2009, The Journal of Neuroscience.

[44]  J. H. Killebrew,et al.  Influence of visual motion on tactile motion perception. , 2006, Journal of neurophysiology.

[45]  Markus Lappe,et al.  Visual selectivity for heading in monkey area MST , 2009, Experimental Brain Research.

[46]  Laurence R. Harris,et al.  Tactile Flow Overrides Other Cues To Self Motion , 2017, Scientific Reports.

[47]  G. DeAngelis,et al.  Multisensory integration: psychophysics, neurophysiology, and computation , 2009, Current Opinion in Neurobiology.

[48]  Neil Burgess,et al.  How vision and self-motion combine or compete during path reproduction changes with age , 2016, Scientific Reports.

[49]  Christopher W. Robinson,et al.  Existence of competing modality dominances , 2016, Attention, Perception, & Psychophysics.

[50]  G. Essick,et al.  Tactile motion activates the human middle temporal/V5 (MT/V5) complex , 2002, The European journal of neuroscience.

[51]  M. Guerraz,et al.  Optimal visuotactile integration for velocity discrimination of self-hand movements. , 2016, Journal of neurophysiology.

[52]  A. Maravita,et al.  The contribution of response conflict, multisensory integration, and body-mediated attention to the crossmodal congruency effect , 2017, Experimental Brain Research.

[53]  Y. Ivanenko,et al.  Space-time relativity in self-motion reproduction. , 2007, Journal of neurophysiology.

[54]  Kenneth H. Britten,et al.  Mechanisms of self-motion perception. , 2008, Annual review of neuroscience.

[55]  K. Hoffmann,et al.  Linear Vestibular Self‐Motion Signals in Monkey Medial Superior Temporal Area , 1999, Annals of the New York Academy of Sciences.

[56]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[57]  Markus Lappe,et al.  Visual selectivity for heading in the macaque ventral intraparietal area. , 2014, Journal of neurophysiology.

[58]  C. Gilbert,et al.  Adult Visual Cortical Plasticity , 2012, Neuron.

[59]  M. Merzenich,et al.  Plasticity in the frequency representation of primary auditory cortex following discrimination training in adult owl monkeys , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[60]  M. Stryker,et al.  Development and Plasticity of the Primary Visual Cortex , 2012, Neuron.

[61]  Monica Gori,et al.  Perceived duration of Visual and Tactile Stimuli Depends on Perceived Speed , 2011, Front. Integr. Neurosci..

[62]  Tony Ro,et al.  Human MST But Not MT Responds to Tactile Stimulation , 2007, The Journal of Neuroscience.

[63]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[64]  Marcia Grabowecky,et al.  Simultaneous shape repulsion and global assimilation in the perception of aspect ratio. , 2011, Journal of vision.

[65]  A. Väljamäe.  Auditorily-induced illusory self-motion: A review , 2009, Brain Research Reviews.

[66]  M Lappe,et al.  Computational mechanisms for optic flow analysis in primate cortex. , 2000, International review of neurobiology.

[67]  Yasushi Ikei,et al.  Tactile Apparent Motion on the Torso Modulates Perceived Forward Self-Motion Velocity , 2016, IEEE Transactions on Haptics.

[68]  Frank Bremmer,et al.  ã Federation of European Neuroscience Societies Heading encoding in the macaque ventral intraparietal area (VIP) , 2022 .

[69]  Shoji Sunaga,et al.  Consistent Air Flow to the Face Facilitates Vection , 2011, Perception.

[70]  Robert A Jacobs,et al.  Bayesian integration of visual and auditory signals for spatial localization. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[71]  D. Angelaki,et al.  Vestibular system: the many facets of a multimodal sense. , 2008, Annual review of neuroscience.

[72]  Olaf Blanke,et al.  Multisensory effects on somatosensation: a trimodal visuo-vestibular-tactile interaction , 2016, Scientific Reports.

[73]  M. Jeannerod,et al.  The role of proprioception in action recognition , 2003, Consciousness and Cognition.

[74]  John J. Foxe,et al.  Multisensory contributions to low-level, ‘unisensory’ processing , 2005, Current Opinion in Neurobiology.

[75]  Peter J Bex,et al.  (In) sensitivity to spatial distortion in natural scenes. , 2010, Journal of vision.

[76]  Kathleen E. Cullen,et al.  The vestibular system: multimodal integration and encoding of self-motion for motor control , 2012, Trends in Neurosciences.

[77]  R. Wurtz,et al.  Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. , 1991, Journal of neurophysiology.

[78]  Leslie G. Ungerleider,et al.  The neural basis of biased competition in human visual cortex , 2001, Neuropsychologia.

[79]  Dora E Angelaki,et al.  Functional Specializations of the Ventral Intraparietal Area for Multisensory Heading Discrimination , 2013, The Journal of Neuroscience.

[80]  S. Sterbing-D’Angelo,et al.  Behavioral/systems/cognitive Multisensory Space Representations in the Macaque Ventral Intraparietal Area , 2022 .

[81]  J. Haxby,et al.  The effect of visual experience on the development of functional architecture in hMT+. , 2007, Cerebral cortex.

[82]  Dora E. Angelaki,et al.  Vestibular Signals in Macaque Extrastriate Visual Cortex Are Functionally Appropriate for Heading Perception , 2009, The Journal of Neuroscience.

[83]  G. DeAngelis,et al.  Representation of Vestibular and Visual Cues to Self-Motion in Ventral Intraparietal Cortex , 2011, The Journal of Neuroscience.

[84]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[85]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[86]  W. Singer,et al.  The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery , 1998, The European journal of neuroscience.

[87]  C. Spence,et al.  Attention and the crossmodal construction of space , 1998, Trends in Cognitive Sciences.

[88]  Olaf Blanke,et al.  Self-motion leads to mandatory cue fusion across sensory modalities. , 2012, Journal of neurophysiology.

[89]  Justine Cléry,et al.  Whole brain mapping of visual and tactile convergence in the macaque monkey , 2015, NeuroImage.

[90]  D. Heeger,et al.  Retinotopy and Functional Subdivision of Human Areas MT and MST , 2002, The Journal of Neuroscience.