The influence of visual cues on temporal anticipation and movement synchronization with musical sequences.

Music presents a complex case of movement timing, as one to several dozen musicians coordinate their actions at short time-scales. This process is often directed by a conductor who provides a visual beat and guides the ensemble through tempo changes. The current experiment tested the ways in which audio-motor coordination is influenced by visual cues from a conductor's gestures, and how this influence might manifest in two ways: movements used to produce sound related to the music, and movements of the upper-body that do not directly affect sound output. We designed a virtual conductor that was derived from morphed motion capture recordings of human conductors. Two groups of participants (29 musicians and 28 nonmusicians, to test the generalizability of visuo-motor synchronization to non-experts) were shown the virtual conductor, a simple visual metronome, or a stationary circle while completing a drumming task that required synchronization with tempo-changing musical sequences. We measured asynchronies and temporal anticipation in the drumming task, as well as participants' upper-body movement using motion capture. Drumming results suggest the conductor generally improves synchronization by facilitating anticipation of tempo changes in the music. Motion capture results showed that the conductor visual cue elicited more structured head movements than the other two visual cues for nonmusicians only. Multiple regression analysis showed that the nonmusicians with less rigid movement and high anticipation had lower asynchronies. Thus, the visual cues provided by a conductor might serve to facilitate temporal anticipation and more synchronous movement in the general population, but might also cause rigid ancillary movements in some non-experts.

[1]  P. Keller,et al.  Sensorimotor synchronization with tempo-changing auditory sequences: Modeling temporal adaptation and anticipation , 2015, Brain Research.

[2]  Benoît G. Bardy,et al.  Coupling of postural activity with motion of a ship at sea , 2015, Experimental Brain Research.

[3]  R. Cañal-Bruland,et al.  Keeping an eye on the violinist: motor experts show superior timing consistency in a visual perception task , 2010, Psychological research.

[4]  J. Ginsborg,et al.  Gestures and glances - the effects of familiarity and expertise on singers’ and pianists’ bodily movements in ensemble rehearsals , 2009 .

[5]  K. Scherer,et al.  Automated Analysis of Body Movement in Emotionally Expressive Piano Performances , 2008 .

[6]  Laurence R. Harris,et al.  The primal role of the vestibular system in determining musical rhythm , 2009, Cortex.

[7]  Ramesh Balasubramaniam,et al.  Two different processes for sensorimotor synchronization in continuous and discontinuous rhythmic movements , 2009, Experimental Brain Research.

[8]  Caroline Palmer,et al.  Temporal coordination and adaptation to rate change in music performance. , 2011, Journal of experimental psychology. Human perception and performance.

[9]  P. Keller,et al.  Individual Differences, Auditory Imagery, and the Coordination of Body Movements and Sounds in Musical Ensembles , 2010 .

[10]  Michael Schutz,et al.  Temporal prediction abilities are mediated by motor effector and rhythmic expertise , 2016, Experimental Brain Research.

[11]  Peter E. Keller,et al.  Multilevel coordination stability: integrated goal representations in simultaneous intra-personal and inter-agent coordination. , 2008, Acta psychologica.

[12]  Marcelo M. Wanderley,et al.  Music and Motion: How Music-Related Ancillary Body Movements Contribute to the Experience of Music , 2009 .

[13]  J. Davidson Bodily movement and facial actions in expressive musical performance by solo and duo instrumentalists: Two distinctive case studies , 2012 .

[14]  B. Repp,et al.  Sensorimotor synchronization: A review of recent research (2006–2012) , 2013, Psychonomic Bulletin & Review.

[15]  Bruno H Repp,et al.  Multiple temporal references in sensorimotor synchronization with metrical auditory sequences , 2007, Psychological research.

[16]  N. Todd,et al.  The sensory-motor theory of rhythm and beat induction 20 years on: a new synthesis and future perspectives , 2015, Front. Hum. Neurosci..

[17]  Michael J. Hove,et al.  The perception of prototypical motion: synchronization is enhanced with quantitatively morphed gestures of musical conductors. , 2012, Journal of experimental psychology. Human perception and performance.

[18]  Benoît G. Bardy,et al.  Getting Your Sea Legs , 2013, PloS one.

[19]  Birgitta Burger,et al.  Hunting for the beat in the body: on period and phase locking in music-induced movement , 2014, Front. Hum. Neurosci..

[20]  K. Kudo,et al.  Variability and fluctuation in running gait cycle of trained runners and non-runners. , 2010, Gait & posture.

[21]  C. Peng,et al.  Mosaic organization of DNA nucleotides. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[22]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[23]  Michael J. Hove,et al.  Spatiotemporal relations and movement trajectories in visuomotor synchronization , 2010 .

[24]  J. Grahn See what I hear? Beat perception in auditory and visual rhythms , 2012, Experimental Brain Research.

[25]  G. Madison,et al.  Effects of practice on variability in an isochronous serial interval production task: asymptotical levels of tapping variability after training are similar to those of musicians. , 2013, Acta psychologica.

[26]  P. Viviani,et al.  The law relating the kinematic and figural aspects of drawing movements. , 1983, Acta psychologica.

[27]  N. Stergiou,et al.  Human movement variability, nonlinear dynamics, and pathology: is there a connection? , 2011, Human movement science.

[28]  Justin London,et al.  Hearing in Time: Psychological Aspects of Musical Meter , 2004 .

[29]  A. Saygin,et al.  The role of appearance and motion in action prediction , 2012, Psychological research.

[30]  Aniruddh D. Patel,et al.  The Beat Alignment Test (BAT): Surveying beat processing abilities in the general population , 2008 .

[31]  Marcelo M. Wanderley,et al.  The role of working memory in the temporal control of discrete and continuous movements , 2014, Experimental Brain Research.

[32]  L. Trainor,et al.  Feeling the Beat: Movement Influences Infant Rhythm Perception , 2005, Science.

[33]  B. Repp A microcosm of musical expression. I. Quantitative analysis of pianists' timing in the initial measures of Chopin's Etude in E major. , 1998, The Journal of the Acoustical Society of America.

[34]  M. T. Elliott,et al.  Multisensory cues improve sensorimotor synchronisation , 2010, The European journal of neuroscience.

[35]  Peter E. Keller,et al.  Adaptation to tempo changes in sensorimotor synchronization: Effects of intention, attention, and awareness , 2004, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[36]  Kristin J. Heaton,et al.  Dynamic visuomotor synchronization: Quantification of predictive timing , 2012, Behavior research methods.

[38]  T. Flash,et al.  Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. , 1995, Journal of experimental psychology. Human perception and performance.

[39]  Jakub S. Sowinski,et al.  Poor synchronization to the beat may result from deficient auditory-motor mapping , 2013, Neuropsychologia.

[40]  P. Keller,et al.  Working memory and auditory imagery predict sensorimotor synchronisation with expressively timed music , 2018, Quarterly journal of experimental psychology.

[41]  S. Kawase Gazing behavior and coordination during piano duo performance , 2013, Attention, Perception, & Psychophysics.

[42]  A. Wing Voluntary Timing and Brain Function: An Information Processing Approach , 2002, Brain and Cognition.

[43]  B. Schultz,et al.  Individual Differences in Temporal Anticipation and Adaptation During Sensorimotor Synchronization , 2015 .

[44]  L. Trainor,et al.  Hearing what the body feels: Auditory encoding of rhythmic movement , 2007, Cognition.

[45]  M. T. Elliott,et al.  Moving in time: Bayesian causal inference explains movement coordination to auditory beats , 2014, Proceedings of the Royal Society B: Biological Sciences.

[46]  Maria A. G. Witek,et al.  Syncopation affects free body-movement in musical groove , 2016, Experimental Brain Research.

[47]  G. Luck,et al.  Ensemble Musicians’ Synchronization With Conductors’ Gestures: An Automated Feature-Extraction Analysis , 2006 .

[48]  Peter E. Keller,et al.  Joint action in music performance , 2008 .

[49]  Michael J Richardson,et al.  Visual tracking and entrainment to an environmental rhythm. , 2007, Journal of experimental psychology. Human perception and performance.

[50]  Bruno H Repp,et al.  Rate Limits in Sensorimotor Synchronization With Auditory and Visual Sequences: The Synchronization Threshold and the Benefits and Costs of Interval Subdivision , 2003, Journal of motor behavior.

[51]  Michael J Richardson,et al.  Influence of stimulus amplitude on unintended visuomotor entrainment. , 2012, Human movement science.

[52]  Jennifer MacRitchie,et al.  Inferring musical structure through bodily gestures , 2013 .

[53]  Chris Davis,et al.  Influence of pacer continuity on continuous and discontinuous visuo-motor synchronisation. , 2016, Acta psychologica.

[54]  L. Craighero,et al.  Leadership in Orchestra Emerges from the Causal Relationships of Movement Kinematics , 2012, PloS one.

[55]  Chien-Chih Wang,et al.  Using detrended fluctuation analysis (DFA) to analyze whether vibratory insoles enhance balance stability for elderly fallers. , 2012, Archives of gerontology and geriatrics.

[56]  D. Sternad,et al.  Variability in motor learning: relocating, channeling and reducing noise , 2009, Experimental Brain Research.

[57]  Marc R. Thompson,et al.  Embodied Meter: Hierarchical Eigenmodes in Music-Induced Movement , 2010 .

[58]  Michael J. Richardson,et al.  Informational constraints on spontaneous visuomotor entrainment. , 2015, Human movement science.

[59]  T. Stoffregen,et al.  Just the sight of you: postural effects of interpersonal visual contact at sea. , 2014, Journal of experimental psychology. Human perception and performance.

[60]  Cordula Vesper,et al.  A minimal architecture for joint action , 2010, Neural Networks.

[61]  Bruno H Repp,et al.  Sensorimotor synchronization and perception of timing: effects of music training and task experience. , 2010, Human movement science.

[62]  Geoff Luck,et al.  An investigation of conductors' temporal gestures and conductor— musician synchronization, and a first experiment , 2008 .

[63]  C. Palmer,et al.  Synchronization of Timing and Motion 435 , 2022 .

[64]  Peter E Keller,et al.  Auditory Pitch Imagery and Its Relationship to Musical Synchronization , 2009, Annals of the New York Academy of Sciences.

[65]  Michael J. Hove,et al.  Interactive Rhythmic Auditory Stimulation Reinstates Natural 1/f Timing in Gait of Parkinson's Patients , 2012, PloS one.

[66]  L. Trainor,et al.  Vestibular influence on auditory metrical interpretation , 2008, Brain and Cognition.

[67]  Pedro Carpena,et al.  Study of the human postural control system during quiet standing using detrended fluctuation analysis , 2009 .

[68]  Tang,et al.  Self-Organized Criticality: An Explanation of 1/f Noise , 2011 .

[69]  G. Knoblich,et al.  Predicting the Effects of Actions: Interactions of Perception and Action , 2001, Psychological science.

[70]  Hugo Merchant,et al.  Finding the beat: a neural perspective across humans and non-human primates , 2015, Philosophical Transactions of the Royal Society B: Biological Sciences.

[71]  S. Havlin,et al.  Detecting long-range correlations with detrended fluctuation analysis , 2001, cond-mat/0102214.

[72]  Michael J. Richardson,et al.  Influence of stimulus velocity profile on rhythmic visuomotor coordination. , 2014, Journal of experimental psychology. Human perception and performance.

[73]  Andreas Daffertshofer,et al.  Keeping with the beat: movement trajectories contribute to movement timing , 2004, Experimental Brain Research.

[74]  R. Schubotz Prediction of external events with our motor system: towards a new framework , 2007, Trends in Cognitive Sciences.

[75]  Peter E. Keller,et al.  The ADaptation and Anticipation Model (ADAM) of sensorimotor synchronization , 2013, Front. Hum. Neurosci..

[76]  Michael J. Richardson,et al.  Rocking together: dynamics of intentional and unintentional interpersonal coordination. , 2007, Human movement science.

[77]  Marc R. Thompson,et al.  Exploring relationships between pianists’ body movements, their expressive intentions, and structural elements of the music , 2012 .

[78]  C. Becchio,et al.  Doing It Your Way: How Individual Movement Styles Affect Action Prediction , 2016, PloS one.

[79]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[80]  I. Peretz,et al.  Born to dance but beat deaf: A new form of congenital amusia , 2011, Neuropsychologia.

[81]  A. Wing,et al.  Optimal feedback correction in string quartet synchronization , 2014, Journal of The Royal Society Interface.

[82]  Michael J. Spivey,et al.  Compatibility of motion facilitates visuomotor synchronization. , 2010, Journal of experimental psychology. Human perception and performance.

[83]  E. Large,et al.  Neural Networks for Beat Perception in Musical Rhythm , 2015, Front. Syst. Neurosci..

[84]  Alan M. Wing,et al.  Perception of string quartet synchronization , 2014, Front. Psychol..

[85]  James M. Hillis,et al.  Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses , 2002, Science.

[86]  Richard B Ivry,et al.  Timing Variability in Circle Drawing and Tapping: Probing the Relationship Between Event and Emergent Timing , 2005, Journal of motor behavior.

[87]  Ruth Rasch,et al.  Synchronization in performed ensemble music , 1979 .

[88]  P. Keller,et al.  The role of temporal prediction abilities in interpersonal sensorimotor synchronization , 2010, Experimental Brain Research.

[89]  Cassie Wilson,et al.  Coordination variability and skill development in expert triple jumpers , 2008, Sports biomechanics.