Meaningful auditory information enhances perception of visual biological motion.

Robust perception requires efficient integration of information from our various senses. Much recent electrophysiology points to neural areas responsive to multisensory stimulation, particularly audiovisual stimulation. However, psychophysical evidence for functional integration of audiovisual motion has been ambiguous. In this study we measure perception of an audiovisual form of biological motion, tap dancing. The results show that the audio tap information interacts with visual motion information, but only when in synchrony, demonstrating a functional combination of audiovisual information in a natural task. The advantage of multimodal combination was better than the optimal maximum likelihood prediction.

[1]  G F Meyer,et al.  The integration of auditory and visual motion signals at threshold , 2003, Perception & psychophysics.

[2]  M. Wallace,et al.  Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. , 1993, Journal of neurophysiology.

[3]  Sophie M. Wuerger,et al.  Low-level integration of auditory and visual motion signals requires spatial co-localisation , 2005, Experimental Brain Research.

[4]  P. Sinha,et al.  Functional neuroanatomy of biological motion perception in humans , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[5]  B. Stein,et al.  Spatial factors determine the activity of multisensory neurons in cat superior colliculus , 1986, Brain Research.

[6]  David Alais,et al.  No direction-specific bimodal facilitation for audiovisual motion detection. , 2004, Brain research. Cognitive brain research.

[7]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[8]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[9]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[10]  M. Alex Meredith,et al.  Neurons and behavior: the same rules of multisensory integration apply , 1988, Brain Research.

[11]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[12]  Edward T. Bullmore,et al.  A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain , 1996, Current Biology.

[13]  R. Blake,et al.  Brain Areas Active during Visual Perception of Biological Motion , 2002, Neuron.

[14]  A. Watson,et al.  Quest: A Bayesian adaptive psychometric method , 1983, Perception & psychophysics.

[15]  Mark T. Wallace,et al.  Chapter 8 The visually responsive neuron and beyond: multisensory integration in cat and monkey , 1993 .

[16]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[17]  Alan C. Evans,et al.  Specific Involvement of Human Parietal Systems and the Amygdala in the Perception of Biological Motion , 1996, The Journal of Neuroscience.

[18]  de Gelder.  Sound Enhances Visual Perception: Cross-Modal Effects of Auditory Organization on Vision , 2001 .

[19]  David Alais,et al.  Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. , 2006, Journal of vision.

[20]  B. Stein,et al.  Spatial determinants of multisensory integration in cat superior colliculus neurons. , 1996, Journal of neurophysiology.

[21]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[22]  L. Fogassi,et al.  Audiovisual mirror neurons and action recognition , 2003, Experimental Brain Research.

[23]  G. Rizzolatti,et al.  Hearing Sounds, Understanding Actions: Action Representation in Mirror Neurons , 2002, Science.

[24]  B. Stein,et al.  The Merging of the Senses , 1993 .

[25]  N. Bolognini,et al.  Enhancement of visual perception by crossmodal visuo-auditory interaction , 2002, Experimental Brain Research.

[26]  M T Wallace,et al.  The visually responsive neuron and beyond: multisensory integration in cat and monkey. , 1993, Progress in brain research.

[27]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[28]  M. Wallace,et al.  Integration of multiple sensory modalities in cat cortex , 2004, Experimental Brain Research.

[29]  Rieko Osu,et al.  The Neural Substrates of Biological Motion Perception: an fMRI Study , 2022 .

[30]  C. Spence,et al.  Attentional capture in serial audiovisual search tasks , 2007, Perception & psychophysics.

[31]  A. Billard,et al.  Auditory motion affects visual biological motion processing , 2007, Neuropsychologia.

[32]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[33]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[34]  M T Wallace,et al.  Development of Multisensory Neurons and Multisensory Integration in Cat Superior Colliculus , 1997, The Journal of Neuroscience.

[35]  C. Keysers,et al.  Empathy and the Somatotopic Auditory Mirror System in Humans , 2006, Current Biology.