Investigating the influence of musical training on cross-modal correspondences and sensorimotor skills in a real-time drawing paradigm

Previous research comparing musically trained and untrained individuals has yielded valuable insights into music cognition and behaviour. Here, we explore two aspects of musical engagement previously studied separately, auditory-visual correspondences and sensorimotor skills, in a novel real-time drawing paradigm. To that end, musically trained and untrained participants were presented with 18 short sequences of pure tones varying in pitch, loudness and tempo, as well as two short musical excerpts. Using an electronic graphics tablet, participants were asked to represent the sound stimuli visually by drawing along with them while they were played. Results revealed that the majority of participants represented pitch with height (higher on the tablet referring to higher pitches), and loudness with the thickness of the line (thicker line for louder sounds). However, musically untrained participants showed a greater diversity of representation strategies and tended to neglect pitch information if unchanged over time. Investigating the performance accuracy in a subgroup of participants revealed that, while pitch-height correspondences were generally represented more accurately than loudness–thickness correspondences, musically trained participants’ representations of pitch and loudness were more accurate. Results are discussed in terms of cross-modal correspondences, the perception of time, and sensorimotor skills.

[1]  Ian Cross,et al.  About Musical Time - Effect of Age, Enjoyment, and Practical Musical Experience on Retrospective Estimate of Elapsed Duration during Music Listening , 2010, COST TD0904 International Workshop.

[2]  Linda B. Smith,et al.  A developmental analysis of the polar structure of dimensions , 1992, Cognitive Psychology.

[3]  Eugenia Costa-Giomi Does Music Instruction Improve Fine Motor Abilities? , 2005, Annals of the New York Academy of Sciences.

[4]  Robert E. Dunn,et al.  Creative Thinking and Music Listening , 1997 .

[5]  T. Matsuzawa,et al.  Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans , 2011, Proceedings of the National Academy of Sciences.

[6]  F. W. Wicker,et al.  Mapping the intersensory regions of perceptual space. , 1968, The American journal of psychology.

[7]  Z. Eitan,et al.  HOW MUSIC MOVES: Musical Parameters and Listeners' Images of Motion , 2006 .

[8]  Daphne Maurer,et al.  Do small white balls squeak? Pitch-object correspondences in young children , 2004, Cognitive, affective & behavioral neuroscience.

[9]  Lawrence M. Zbikowski Conceptualizing Music: Cognitive Structure, Theory, and Analysis , 2002 .

[10]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[11]  R Walker,et al.  The effects of culture, environment, age, and musical training on choices of visual metaphors for sound , 1987, Perception & psychophysics.

[12]  R. Butler,et al.  Factors that influence the localization of sound in the vertical plane. , 1968, The Journal of the Acoustical Society of America.

[13]  Bruno H. Repp,et al.  Tapping to a Very Slow Beat: A Comparison of Musicians and Nonmusicians , 2007 .

[14]  Robert J. Zatorre,et al.  Moving on Time: Brain Network for Auditory-Motor Synchronization is Modulated by Rhythm Complexity and Musical Training , 2008, Journal of Cognitive Neuroscience.

[15]  L. Marks,et al.  Cross-modality matching of brightness and loudness. , 1965, Proceedings of the National Academy of Sciences of the United States of America.

[16]  C. Krumhansl,et al.  Cross-modal interactions in the perception of musical performance , 2006, Cognition.

[17]  M. Brass,et al.  Imitation: is cognitive neuroscience solving the correspondence problem? , 2005, Trends in Cognitive Sciences.

[18]  Donna Coch,et al.  Music training and working memory: An ERP study , 2011, Neuropsychologia.

[19]  C. C. Pratt The spatial character of high and low tones. , 1930 .

[20]  L E Marks,et al.  On associations of light and sound: the mediation of brightness, pitch, and loudness. , 1974, The American journal of psychology.

[21]  Lauren Stewart,et al.  Reading music modifies spatial mapping in pianists , 2004, Perception & psychophysics.

[22]  S. Mudd,et al.  SPATIAL STEREOTYPES OF FOUR DIMENSIONS OF PURE TONE. , 1963, Journal of experimental psychology.

[23]  Jim Tørresen,et al.  Analyzing sound tracings: a multimodal approach to music information retrieval , 2011, MIRUM '11.

[24]  Scott P. Johnson,et al.  Preverbal Infants’ Sensitivity to Synaesthetic Cross-Modality Correspondences , 2010, Psychological science.

[25]  C. Spence Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[26]  David J. Lewkowicz,et al.  Cross-modal equivalence in early infancy: Auditory–visual intensity matching. , 1980 .

[27]  Ellen Winner,et al.  "Metaphorical" Mapping in Human Infants , 1981 .

[28]  C. Spence,et al.  Crossmodal correspondences: Innate or learned? , 2012, i-Perception.

[29]  Eric Clarke,et al.  Ways of Listening , 2005 .

[30]  Stephen W. Smoliar,et al.  The Mind Behind the Musical Ear: How Children Develop Musical , 1995 .

[31]  S. S. Stevens,et al.  Cross-modality matching of brightness to loudness by 5-year-olds , 1969 .

[32]  L. Marks Bright sneezes and dark coughs, loud sunlight and soft moonlight. , 1982, Journal of experimental psychology. Human perception and performance.

[33]  Yôiti Suzuki,et al.  Equal-loudness-level contours for pure tones. , 2004, The Journal of the Acoustical Society of America.

[34]  J. Morais,et al.  Spatial associations for musical stimuli: a piano in the head? , 2007, Journal of experimental psychology. Human perception and performance.

[35]  R. Mccall,et al.  The Genetic and Environmental Origins of Learning Abilities and Disabilities in the Early School , 2007, Monographs of the Society for Research in Child Development.

[36]  Dan Tidhar,et al.  Synaesthetic Traces: Digital Acquisition of Musical Shapes , 2011 .

[37]  M. Ernst Learning to integrate arbitrary signals from vision and touch. , 2007, Journal of vision.

[38]  Tal Savion-Lemieux,et al.  The effect of early musical training on adult motor performance: evidence for a sensitive period in motor learning , 2006, Experimental Brain Research.

[39]  Renee Timmers,et al.  Beethoven’s last piano sonata and those who follow crocodiles: Cross-domain mappings of auditory pitch in a musical context , 2010, Cognition.

[40]  R. Butler,et al.  Localization of tonal stimuli in the vertical plane. , 1968, The Journal of the Acoustical Society of America.

[41]  S. Bennett,et al.  Developmental effects influencing observational modelling: A meta-analysis , 2007, Journal of sports sciences.

[42]  Bruno L. Giordano,et al.  Spatial representation of pitch height: the SMARC effect , 2006, Cognition.

[43]  Siu-Lan Tan,et al.  Graphic Representations of Short Musical Compositions , 2004 .

[44]  Virginia B. Penhune,et al.  Gesture imitation in musicians and non-musicians , 2010, Experimental Brain Research.

[45]  P. Boersma Praat : doing phonetics by computer (version 5.1.05) , 2009 .

[46]  Zohar Eitan,et al.  How pitch and loudness shape musical space and motion , 2013 .

[47]  M. Sa,et al.  SPATIAL STEREOTYPES OF FOUR DIMENSIONS OF PURE TONE. , 1963 .

[48]  Evan R. Powell,et al.  Influences of Musical Variables on Pictorial Connotations. , 1970 .

[49]  Nicholas Cook,et al.  Music, Imagination, And Culture , 1991 .

[50]  R. Zatorre,et al.  When the brain plays music: auditory–motor interactions in music perception and production , 2007, Nature Reviews Neuroscience.

[51]  Scott D. Lipscomb,et al.  PERCEIVED MATCH BETWEEN VISUAL PARAMETERS AND AUDITORY CORRELATES: AN EXPERIMENTAL MULTIMEDIA INVESTIGATION , 2004 .

[52]  Emery Schubert Modeling Perceived Emotion With Continuous Musical Features , 2004 .

[53]  Rena Upitis,et al.  Figural and Metric Understanding of Rhythm , 1994 .

[54]  Daniel Leech-Wilkinson Making music with Alfred Cortot: ontology, data, analysis , 2011 .

[55]  Eric Clarke,et al.  Ways of Listening: An Ecological Approach to the Perception of Musical Meaning , 2005 .

[56]  Bruno H Repp,et al.  Sensorimotor synchronization and perception of timing: effects of music training and task experience. , 2010, Human movement science.

[57]  André Dufour,et al.  Effect of musical expertise on visuospatial abilities: Evidence from reaction times and mental imagery , 2004, Brain and Cognition.

[58]  J. Ramsay,et al.  Introduction to Functional Data Analysis , 2007 .

[59]  L. Nygaard,et al.  The Semantics of Prosody: Acoustic and Perceptual Evidence of Prosodic Correlates to Word Meaning , 2009, Cogn. Sci..

[60]  Siu-Lan Tan,et al.  Musically untrained college students' interpretations of musical notation: sound, silence, loudness, duration, and temporal order , 2009 .

[61]  John T. Cowles,et al.  An experimental study of the pairing of certain auditory and visual stimuli. , 1935 .