Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony

We investigated the effect of musical expertise on sensitivity to asynchrony for drumming point-light displays, which varied in their physical characteristics (Experiment 1) or in their degree of audiovisual congruency (Experiment 2). In Experiment 1, 21 repetitions of three tempos × three accents × nine audiovisual delays were presented to four jazz drummers and four novices. In Experiment 2, ten repetitions of two audiovisual incongruency conditions × nine audiovisual delays were presented to 13 drummers and 13 novices. Participants gave forced-choice judgments of audiovisual synchrony. The results of Experiment 1 show an enhancement in experts’ ability to detect asynchrony, especially for slower drumming tempos. In Experiment 2 an increase in sensitivity to asynchrony was found for incongruent stimuli; this increase, however, is attributable only to the novice group. Altogether the results indicated that through musical practice we learn to ignore variations in stimulus characteristics that otherwise would affect our multisensory integration processes.

[1]  Charles Spence,et al.  Temporal recalibration during asynchronous audiovisual speech perception , 2007, Experimental Brain Research.

[2]  V. Torre,et al.  Sensory Transduction , 1990, NATO ASI Series.

[3]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[4]  H. McGurk,et al.  Visual influences on speech perception processes , 1978, Perception & psychophysics.

[5]  Ralf Steinmetz,et al.  Human Perception of Jitter and Media Synchronization , 1996, IEEE J. Sel. Areas Commun..

[6]  Robert Sekuler,et al.  Multi-sensory integration of spatio-temporal segmentation cues: one plus one does not always equal two , 2007, Experimental Brain Research.

[7]  P F Seitz,et al.  The use of visible speech cues for improving auditory detection of spoken sentences. , 2000, The Journal of the Acoustical Society of America.

[8]  C. Spence,et al.  Audiovisual synchrony perception for speech and music assessed using a temporal order judgment task , 2006, Neuroscience Letters.

[9]  A. King,et al.  Multisensory Integration: Strategies for Synchronization , 2005, Current Biology.

[10]  C. Krumhansl,et al.  Mental representations for musical meter. , 1990, Journal of experimental psychology. Human perception and performance.

[11]  D W Massaro,et al.  Perception of asynchronous and conflicting visual and auditory speech. , 1996, The Journal of the Acoustical Society of America.

[12]  Michael Peter Hollier,et al.  An Experimental Investigation into Multi-Modal Synchronization Sensitivity for Perceptual Model Development , 1998 .

[13]  Waka Fujisaki,et al.  Temporal frequency characteristics of synchrony–asynchrony discrimination of audio-visual signals , 2005, Experimental Brain Research.

[14]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[15]  Dominic W. Massaro,et al.  Erratum: “Perception of asynchronous and conflicting visual and auditory speech’’ [J. Acoust. Soc. Am. 100, 1777–1786 (1996)] , 1997 .

[16]  Michael Kubovy,et al.  The effect of tone envelope on sensory integration: support for the 'unity assumption' , 2008 .

[17]  James V. Stone,et al.  When is now? Perception of simultaneity , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[18]  David Poeppel,et al.  Detection of auditory (cross-spectral) and auditory-visual (cross-modal) synchrony , 2004, Speech Commun..

[19]  Charles Spence,et al.  Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. , 2005, Brain research. Cognitive brain research.

[20]  Jeroen J. A. van Boxtel,et al.  Retinotopic and non-retinotopic stimulus encoding in binocular rivalry and the involvement of feedback. , 2008, Journal of vision.

[21]  W. Hairston,et al.  Aspects of Multisensory Perception: The Integration of Visual and Auditory Information in Musical Experiences , 2005, Annals of the New York Academy of Sciences.

[22]  J. Sloboda,et al.  An Investigation of Musicians’ Synchronization with Traditional Conducting Beat Patterns , 2010 .

[23]  E. Saltzman,et al.  Space-time behavior of single and bimanual rhythmical movements: data and limit cycle model. , 1987 .

[24]  D. Moelants.  Preferred tempo reconsidered. , 2002 .

[25]  Charles Spence,et al.  On the inability to ignore touch when responding to vision in the crossmodal congruency task. , 2005, Acta psychologica.

[26]  C. Spence,et al.  Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli , 2007, Perception & psychophysics.

[27]  Davide Rocchesso,et al.  COMPUTATION OF NONLINEAR FILTER NETWORKS CONTAINING DELAY-FREE PATHS , 2004 .

[28]  J. Koenderink,et al.  A new twist to the “shading cue” , 2010 .

[29]  D. Poeppel,et al.  Temporal window of integration in auditory-visual speech perception , 2007, Neuropsychologia.

[30]  Carl Haakon Waadeland.  Strategies in empirical studies of swing groove , 2006 .

[31]  Hamish G MacDougall,et al.  Marching to the beat of the same drummer: the spontaneous tempo of human locomotion. , 2005, Journal of applied physiology.

[32]  C. Spence,et al.  Audiovisual temporal order judgments , 2003, Experimental Brain Research.

[33]  Steven Greenberg,et al.  The temporal properties of spoken Japanese are similar to those of English , 1997, EUROSPEECH.

[34]  M. R. Jones.  Dynamic pattern structure in music: Recent theory and research , 1987, Perception & psychophysics.

[35]  Geoff Luck,et al.  An investigation of conductors' temporal gestures and conductor— musician synchronization, and a first experiment , 2008 .

[36]  Stefan Skare,et al.  See Blockindiscussions, Blockinstats, Blockinand Blockinauthor Blockinprofiles Blockinfor Blockinthis Blockinpublication Extensive Blockinpiano Blockinpracticing Blockinhas Blockinregionally Specific Blockineffects Blockinon Blockinwhite Blockinmatter Blockindevelopment , 2022 .

[37]  A. Wagner,et al.  Analysis of Drumbeats – Interaction between Drummer , Drumstick and Instrument , 2005 .

[38]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[39]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[40]  A. Billard,et al.  Auditory motion affects visual biological motion processing , 2007, Neuropsychologia.

[41]  C. Spence,et al.  Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli. , 2008, Acta psychologica.

[42]  A. J. King,et al.  Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus , 2004, Experimental Brain Research.

[43]  Yôiti Suzuki,et al.  Implicit estimation of sound-arrival time , 2003, Nature.

[44]  C. Drake,et al.  The development of rhythmic attending in auditory sequences: attunement, referent period, focal attending , 2000, Cognition.

[45]  G. Schlaug,et al.  Brain Structures Differ between Musicians and Non-Musicians , 2003, The Journal of Neuroscience.

[46]  P. Gribble,et al.  Temporal constraints on the McGurk effect , 1996, Perception & psychophysics.

[47]  A. B. Drought,et al.  WALKING PATTERNS OF NORMAL MEN. , 1964, The Journal of bone and joint surgery. American volume.

[48]  C. Spence,et al.  Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgments , 2008, Experimental Brain Research.

[49]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[50]  C. Drake,et al.  Tempo sensitivity in auditory sequences: Evidence for a multiple-look model , 1993, Perception & psychophysics.

[51]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[52]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[53]  Thomas P. Caudell,et al.  Computational Requirements and Synchronization Issues for Virtual Acoustic Displays , 1998, Presence.

[54]  Michael Schutz,et al.  Hearing Gestures, Seeing Music: Vision Influences Perceived Tone Duration , 2007, Perception.

[55]  F. Pollick,et al.  When knowing can replace seeing in audiovisual integration of actions , 2009, Cognition.

[56]  Steven Greenberg,et al.  Speech intelligibility derived from asynchronous processing of auditory-visual information , 2001, AVSP.

[57]  C. Spence,et al.  Multisensory prior entry. , 2001, Journal of experimental psychology. General.

[58]  Robert J Zatorre,et al.  Differences in Gray Matter between Musicians and Nonmusicians , 2005, Annals of the New York Academy of Sciences.

[59]  Ayse Pinar Saygin,et al.  In the Footsteps of Biological Motion and Multisensory Perception , 2008, Psychological science.

[60]  Derek H. Arnold,et al.  A paradox of temporal perception revealed by a stimulus oscillating in colour and orientation , 2003, Vision Research.

[61]  Asif A Ghazanfar,et al.  Facilitation of multisensory integration by the "unity effect" reveals that speech is special. , 2008, Journal of vision.

[62]  Johan Sundberg,et al.  TIME DISCRIMINATION IN A MONOTONIC, ISOCHRONOUS SEQUENCE , 1995 .

[63]  R. Church,et al.  Preferred rates of repetitive tapping and categorical time production , 1994, Perception & psychophysics.

[64]  David Alais,et al.  Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. , 2006, Journal of vision.

[65]  C. Spence,et al.  Audiovisual synchrony perception for music, speech, and object actions , 2006, Brain Research.

[66]  Sofia Dahl,et al.  Playing the accent : comparing striking velocity and timing in an ostinato rhythm performed by four drummers , 2004 .