In the Footsteps of Biological Motion and Multisensory Perception

Observers judged whether a periodically moving visual display (point-light walker) had the same temporal frequency as a series of auditory beeps that in some cases coincided with the apparent footsteps of the walker. Performance in this multisensory judgment was consistently better for upright point-light walkers than for inverted point-light walkers or scrambled control stimuli, even though the temporal information was the same in the three types of stimuli. The advantage with upright walkers disappeared when the visual “footsteps” were not phase-locked with the auditory events (and instead offset by 50% of the gait cycle). This finding indicates there was some specificity to the naturally experienced multisensory relation, and that temporal perception was not simply better for upright walkers per se. These experiments indicate that the gestalt of visual stimuli can substantially affect multisensory judgments, even in the context of a temporal task (for which audition is often considered dominant). This effect appears to be constrained by the ecological validity of the particular pairings.

[1]  Randolph Blake,et al.  Eccentric perception of biological motion is unscalably poor , 2005, Vision Research.

[2]  Felix Wichmann,et al.  The psychometric function: II. Bootstrap-based confidence intervals and sampling , 2001, Perception & psychophysics.

[3]  Michael Schutz,et al.  Hearing Gestures, Seeing Music: Vision Influences Perceived Tone Duration , 2007, Perception.

[4]  A. Billard,et al.  Auditory motion affects visual biological motion processing , 2007, Neuropsychologia.

[5]  R. Blake,et al.  What constitutes an efficient reference frame for vision? , 2002, Nature Neuroscience.

[6]  M. Pavlova,et al.  Orientation specificity in biological motion perception , 2000, Perception & psychophysics.

[7]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.

[8]  R. Blake,et al.  Brain Areas Active during Visual Perception of Biological Motion , 2002, Neuron.

[9]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[10]  S. Shimojo,et al.  Visual illusion induced by sound. , 2002, Brain research. Cognitive brain research.

[11]  Chris I. Baker,et al.  Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions , 2005, Journal of Cognitive Neuroscience.

[12]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[13]  T. Bachmann,et al.  Cognitive contributions to the perception of spatial and temporal events (Advances in Psychology, Vol. 129) , 1999 .

[14]  de Gelder Sound Enhances Visual Perception: Cross-Modal Effects of Auditory Organization on Vision , 2001 .

[15]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .

[16]  R. Blake,et al.  Perception of human motion. , 2007, Annual review of psychology.

[17]  N. Troje,et al.  The Inversion Effect in Biological Motion Perception: Evidence for a “Life Detector”? , 2006, Current Biology.

[18]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[19]  N. Birbaumer,et al.  Dissociable cortical processing of recognizable and non-recognizable biological movement: analysing gamma MEG activity. , 2004, Cerebral cortex.

[20]  Simon Carlile,et al.  Synchronizing to real events: subjective audiovisual alignment scales with perceived auditory depth and speed of sound. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[21]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[22]  D R Proffitt,et al.  The development of infant sensitivity to biomechanical motions. , 1985, Child development.

[23]  R. Welch Chapter 15 Meaning, attention, and the “unity assumption” in the intersensory bias of spatial and temporal perceptions , 1999 .

[24]  G. Johansson Visual perception of biological motion and a model for its analysis , 1973 .

[25]  T. Shipley The Effect of Object and Event Orientation on Perception of Biological Motion , 2003, Psychological science.

[26]  D. Lewkowicz,et al.  Learning and discrimination of audiovisual events in human infants: the hierarchical relation between intersensory temporal synchrony and rhythmic pattern cues. , 2003, Developmental psychology.

[27]  Stefanie Hoehl,et al.  The perception of biological motion by infants: An event-related potential study , 2006, Neuroscience Letters.

[28]  A. Saygin Superior temporal and premotor brain areas necessary for biological motion perception. , 2007, Brain : a journal of neurology.

[29]  Anna Oleksiak,et al.  The influence of biological motion perception on structure-from-motion interpretations at different speeds. , 2006, Journal of vision.

[30]  C. Spence,et al.  The Handbook of Multisensory Processing , 2004 .

[31]  D R Proffitt,et al.  Perception of biomechanical motions by infants: implementation of various processing constraints. , 1987, Journal of experimental psychology. Human perception and performance.

[32]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[33]  James E. Cutting,et al.  A program to generate synthetic walkers as dynamic point-light displays , 1978 .

[34]  Nikolaus F. Troje,et al.  Point-light walkers with and without local motion features for determining direction , 2007 .

[35]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[36]  C. Spence,et al.  Audiovisual prior entry , 2003, Neuroscience Letters.

[37]  M. Sereno,et al.  Point-Light Biological Motion Perception Activates Human Premotor Cortex , 2004, The Journal of Neuroscience.

[38]  Elizabeth S Spelke,et al.  Perception of Moving, Sounding Objects by Four-Month-Old Infants , 1983, Perception.

[39]  David Alais,et al.  Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. , 2006, Journal of vision.

[40]  Randolph Blake,et al.  Hearing What the Eyes See , 2005, Psychological science.

[41]  S. Sumi Upside-down Presentation of the Johansson Moving Light-Spot Pattern , 1984, Perception.