Long-term music training tunes how the brain temporally binds signals from multiple senses

Practicing a musical instrument is a rich multisensory experience involving the integration of visual, auditory, and tactile inputs with motor responses. This combined psychophysics–fMRI study used the musician's brain to investigate how sensory-motor experience molds temporal binding of auditory and visual signals. Behaviorally, musicians exhibited a narrower temporal integration window than nonmusicians for music but not for speech. At the neural level, musicians showed increased audiovisual asynchrony responses and effective connectivity selectively for music in a superior temporal sulcus-premotor-cerebellar circuitry. Critically, the premotor asynchrony effects predicted musicians’ perceptual sensitivity to audiovisual asynchrony. Our results suggest that piano practicing fine tunes an internal forward model mapping from action plans of piano playing onto visible finger movements and sounds. This internal forward model furnishes more precise estimates of the relative audiovisual timings and hence, stronger prediction error signals specifically for asynchronous music in a premotor-cerebellar circuitry. Our findings show intimate links between action production and audiovisual temporal binding in perception.

[1]  Davide Rocchesso,et al.  Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony , 2009, Experimental Brain Research.

[2]  M. Hallett,et al.  Neural correlates of cross-modal binding , 2003, Nature Neuroscience.

[3]  Hans-Jochen Heinze,et al.  Shared networks for auditory and motor processing in professional pianists: Evidence from fMRI conjunction , 2006, NeuroImage.

[4]  C. Caltagirone,et al.  Repetitive TMS of cerebellum interferes with millisecond time processing , 2007, Experimental Brain Research.

[5]  H. Zelaznik,et al.  Disrupted Timing of Discontinuous But Not Continuous Movements by Cerebellar Lesions , 2003, Science.

[6]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[7]  R. Zatorre,et al.  Moving on Time: Brain Network for Auditory-Motor Synchronization is Modulated by Rhythm Complexity and Musical Training , 2008, Journal of Cognitive Neuroscience.

[8]  R. Miall,et al.  Brain activation patterns during measurement of sub- and supra-second intervals , 2003, Neuropsychologia.

[9]  R. Zatorre,et al.  Listening to musical rhythms recruits motor regions of the brain. , 2008, Cerebral cortex.

[10]  Giacomo Koch,et al.  Role of the cerebellum in externally paced rhythmic finger movements. , 2007, Journal of neurophysiology.

[11]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[12]  Richard B. Ivry,et al.  Cerebellar activation during discrete and not continuous timed movements: An fMRI study , 2007, NeuroImage.

[13]  E. Saltzman,et al.  Action Representation of Sound: Audiomotor Recognition Network While Listening to Newly Acquired Actions , 2007, The Journal of Neuroscience.

[14]  Karl J. Friston,et al.  Bayesian model selection for group studies , 2009, NeuroImage.

[15]  U. Noppeney,et al.  Distinct Functional Contributions of Primary Sensory and Association Areas to Audiovisual Integration in Object Categorization , 2010, The Journal of Neuroscience.

[16]  Alan C. Evans,et al.  Cerebellar Contributions to Motor Timing: A PET Study of Auditory and Visual Rhythm Reproduction , 1998, Journal of Cognitive Neuroscience.

[17]  A. Nobre,et al.  The Cerebellum Predicts the Timing of Perceptual Events , 2008, The Journal of Neuroscience.

[18]  Conny F. Schmidt,et al.  A network for audio–motor coordination in skilled pianists and non-musicians , 2007, Brain Research.

[19]  Nicole M. Russo,et al.  Musical experience shapes human brainstem encoding of linguistic pitch patterns , 2007, Nature Neuroscience.

[20]  R. Ivry,et al.  Detecting violations of sensory expectancies following cerebellar degeneration: A mismatch negativity study , 2008, Neuropsychologia.

[21]  Sibylle C. Herholz,et al.  Cortical Plasticity Induced by Short-Term Unimodal and Multimodal Musical Training , 2008, The Journal of Neuroscience.

[22]  Fredrik Ullén,et al.  Dissociation between melodic and rhythmic processing during piano performance from musical scores , 2006, NeuroImage.

[23]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[24]  N. Ramnani The primate cortico-cerebellar system: anatomy and function , 2006, Nature Reviews Neuroscience.

[25]  R. Zatorre,et al.  When the brain plays music: auditory–motor interactions in music perception and production , 2007, Nature Reviews Neuroscience.

[26]  E. Altenmüller,et al.  Transmodal Sensorimotor Networks during Action Observation in Professional Pianists , 2005, Journal of Cognitive Neuroscience.

[27]  N. Kraus,et al.  Music training for the development of auditory skills , 2010, Nature Reviews Neuroscience.

[28]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[29]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[30]  M. Sams,et al.  Musicians have enhanced subcortical auditory and audiovisual processing of speech and music , 2007, Proceedings of the National Academy of Sciences.

[31]  Lee M. Miller,et al.  Behavioral/systems/cognitive Perceptual Fusion and Stimulus Coincidence in the Cross- Modal Integration of Speech , 2022 .

[32]  J. Binder,et al.  Distributed Neural Systems Underlying the Timing of Movements , 1997, The Journal of Neuroscience.

[33]  David H. Foster,et al.  Model-free estimation of the psychometric function , 2010 .

[34]  E. Altenmüller,et al.  The musician's brain as a model of neuroplasticity , 2002, Nature Reviews Neuroscience.

[35]  Albert R. Powers,et al.  Perceptual Training Narrows the Temporal Window of Multisensory Binding , 2009, The Journal of Neuroscience.

[36]  U. Noppeney,et al.  Audiovisual Synchrony Improves Motion Discrimination via Enhanced Connectivity between Early Visual and Auditory Areas , 2010, The Journal of Neuroscience.

[37]  R. Ivry,et al.  The neural representation of time , 2004, Current Opinion in Neurobiology.

[38]  D. Buonomano,et al.  The neural basis of temporal processing. , 2004, Annual review of neuroscience.

[39]  Hans Forssberg,et al.  Listening to rhythms activates motor and premotor cortices , 2009, Cortex.

[40]  Jessica A. Grahn,et al.  Feeling the Beat: Premotor and Striatal Interactions in Musicians and Nonmusicians during Beat Perception , 2009, The Journal of Neuroscience.

[41]  Robert J. Zatorre,et al.  Interactions between auditory and dorsal premotor cortex during synchronization to musical rhythms , 2006, NeuroImage.

[42]  Karl J. Friston,et al.  Dynamic causal modeling , 2010, Scholarpedia.

[43]  Aniruddh D. Patel,et al.  The linguistic benefits of musical abilities , 2007, Trends in Cognitive Sciences.

[44]  Karl J. Friston,et al.  Comparing Families of Dynamic Causal Models , 2010, PLoS Comput. Biol..

[45]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.