Movement sonification: Effects on perception and action

Research shows that it enhances users' perception to present stimuli in two modalities simultaneously (for example, with speech perception). In our experiments, we use sonification to transform parameters of human movement patterns into sound to enhance perception accuracy. This article also presents further features of the human perceptual system like multisensory integration and perceptual stream dissociation with regard to effective interactive sonification.

[1]  Alfred O. Effenberg.  MULTIMODAL CONVERGENT INFORMATION ENHANCES PERCEPTION ACCURACY OF HUMAN MOVEMENT PATTERNS , 2001 .

[2]  Arlene S. Walker-Andrews.  Infants, Too, are Global Perceivers (Commentary on T.A. Stoffregen and B.G. Bardy, On Specification and the Senses) , 2001 .

[3]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[4]  D. Massaro.  Perceiving talking faces: from speech perception to a behavioral principle , 1999 .

[5]  Jon H. Kaas,et al.  'What' and 'where' processing in auditory cortex , 1999, Nature Neuroscience.

[6]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[7]  Michael J Brammer,et al.  Crossmodal identification , 1998, Trends in Cognitive Sciences.

[8]  T. Stoffregen,et al.  On specification and the senses , 2001, Behavioral and Brain Sciences.

[9]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[10]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[11]  W. Dunlap,et al.  Meta-Analysis of Experiments With Matched Groups or Repeated Measures Designs , 1996 .

[12]  S. Runeson,et al.  Kinematic specification of dynamics as an informational basis for person and action perception: Expe , 1983 .

[13]  T. Takeuchi,et al.  Auditory Information in Playing Tennis , 1993, Perceptual and motor skills.

[14]  B. Stein.  Neural mechanisms for synthesizing sensory information and producing adaptive behaviors , 1998, Experimental Brain Research.

[15]  D. Burr,et al.  A cortical area that responds specifically to optic flow, revealed by fMRI , 2000, Nature Neuroscience.

[16]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[17]  Mahmood R. Azimi-Sadjadi,et al.  Rhythmic finger tapping to cosine-wave modulated metronome sequences: Evidence of subliminal entrainment , 1998 .

[18]  Gregory Kramer,et al.  An introduction to auditory display , 1994 .

[19]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[20]  R A Andersen,et al.  Multimodal integration for the representation of space in the posterior parietal cortex. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[21]  R. Blake,et al.  Brain Areas Active during Visual Perception of Biological Motion , 2002, Neuron.

[22]  D. Massaro,et al.  Perceiving Talking Faces , 1995 .