Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation

The ability to anticipate forthcoming events has clear evolutionary advantages, and predictive successes or failures often entail significant psychological and physiological consequences. In music perception, the confirmation and violation of expectations are critical to the communication of emotion and aesthetic effects of a composition. Neuroscientific research on musical expectations has focused on harmony. Although harmony is important in Western tonal styles, other musical traditions, emphasizing pitch and melody, have been rather neglected. In this study, we investigated melodic pitch expectations elicited by ecologically valid musical stimuli by drawing together computational, behavioural, and electrophysiological evidence. Unlike rule-based models, our computational model acquires knowledge through unsupervised statistical learning of sequential structure in music and uses this knowledge to estimate the conditional probability (and information content) of musical notes. Unlike previous behavioural paradigms that interrupt a stimulus, we devised a new paradigm for studying auditory expectation without compromising ecological validity. A strong negative correlation was found between the probability of notes predicted by our model and the subjectively perceived degree of expectedness. Our electrophysiological results showed that low-probability notes, as compared to high-probability notes, elicited a larger (i) negative ERP component at a late time period (400-450 ms), (ii) beta band (14-30 Hz) oscillation over the parietal lobe, and (iii) long-range phase synchronization between multiple brain regions. Altogether, the study demonstrated that statistical learning produces information-theoretic descriptions of musical notes that are proportional to their perceived expectedness and are associated with characteristic patterns of neural activity.

[1]  Eugene Narmour,et al.  The Analysis and Cognition of Basic Melodic Structures: The Implication-Realization Model , 1990 .

[2]  Ian H. Witten,et al.  Multiple viewpoint systems for music prediction , 1995 .

[3]  Aniruddh D. Patel,et al.  Top‐Down Control of Rhythm Perception Modulates Early Auditory Responses , 2009, Annals of the New York Academy of Sciences.

[4]  J. H. Steiger Tests for comparing elements of a correlation matrix. , 1980 .

[5]  W. Jay Dowling,et al.  Simplicity and complexity in music and cognition , 1989 .

[6]  Aniruddh D. Patel,et al.  Processing Syntactic Relations in Language and Music: An Event-Related Potential Study , 1998, Journal of Cognitive Neuroscience.

[7]  H. Petsche,et al.  Long-Range Synchrony in the γ Band: Role in Music Perception , 2001, The Journal of Neuroscience.

[8]  E. Schellenberg,et al.  Simplifying the Implication-Realization Model of Melodic Expectancy , 1997 .

[9]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[10]  P. Fries A mechanism for cognitive dynamics: neuronal communication through neuronal coherence , 2005, Trends in Cognitive Sciences.

[11]  T. Lagerlund,et al.  Interelectrode coherences from nearest-neighbor and spherical harmonic expansion computation of laplacian of scalp potential. , 1993, Electroencephalography and clinical neurophysiology.

[12]  Uffe K. Wiil,et al.  Computer Music Modelling and Retrieval , 2004 .

[13]  E. Ziegel Permutation, Parametric, and Bootstrap Tests of Hypotheses (3rd ed.) , 2005 .

[14]  R. Carlyon,et al.  An Information Theoretic Characterisation of Auditory Encoding , 2007, PLoS biology.

[15]  D. Tucker,et al.  EEG coherency. I: Statistics, reference electrode, volume conduction, Laplacians, cortical imaging, and interpretation at multiple scales. , 1997, Electroencephalography and clinical neurophysiology.

[16]  F. Varela,et al.  Perception's shadow: long-distance synchronization of human brain activity , 1999, Nature.

[17]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[18]  Geraint A. Wiggins,et al.  Improved Methods for Statistical Modelling of Monophonic Music , 2004 .

[19]  R. Jackendoff,et al.  A Generative Theory of Tonal Music , 1985 .

[20]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[21]  G. Ermentrout,et al.  Gamma rhythms and beta rhythms have different synchronization properties. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[22]  L. Basile,et al.  Complex, multifocal, individual-specific attention-related cortical functional circuits. , 2007, Biological research.

[23]  A. Friederici,et al.  Musical syntax is processed in Broca's area: an MEG study , 2001, Nature Neuroscience.

[24]  P. Good Permutation, Parametric, and Bootstrap Tests of Hypotheses , 2005 .

[25]  Hiroshi Nittono,et al.  Event-related potentials elicited by wrong terminal notes: effects of temporal disruption , 2000, Biological Psychology.

[26]  E. Schellenberg,et al.  Expectancy in melody: tests of the implication-realization model , 1996, Cognition.

[27]  F. L. D. Silva,et al.  Beta rebound after different types of motor imagery in man , 2005, Neuroscience Letters.

[28]  Christo Pantev,et al.  Neural basis of music imagery and the effect of musical expertise , 2008, The European journal of neuroscience.

[29]  J. Snyder,et al.  Gamma-band activity reflects the metric structure of rhythmic tone sequences. , 2005, Brain research. Cognitive brain research.

[30]  M. Tervaniemi,et al.  Representation of harmony rules in the human brain: Further evidence from event-related potentials , 2007, Brain Research.

[31]  F. Varela,et al.  Measuring phase synchrony in brain signals , 1999, Human brain mapping.

[32]  Stefan Koelsch,et al.  Toward the Neural Basis of Processing Structure in Music , 2003, Annals of the New York Academy of Sciences.

[33]  G. R. Muller,et al.  Event-related beta EEG changes during wrist movements induced by functional electrical stimulation of forearm muscles in man , 2003, Neuroscience Letters.

[34]  Mireille Besson,et al.  Visually Induced Auditory Expectancy in Music Reading: A Behavioral and Electrophysiological Study , 2005, Journal of Cognitive Neuroscience.

[35]  Rodrigo Quian Quiroga,et al.  Nonlinear multivariate analysis of neurophysiological signals , 2005, Progress in Neurobiology.

[36]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[37]  A. Friederici,et al.  Brain Indices of Music Processing: Nonmusicians are Musical , 2000, Journal of Cognitive Neuroscience.

[38]  安井 拓也 Language and music : differential hemispheric dominance in detecting unexpected errors in the lyrics and melody of memorized songs , 2008 .

[39]  R. Hari,et al.  Functional Segregation of Movement-Related Rhythmic Activity in the Human Brain , 1995, NeuroImage.

[40]  Geraint A. Wiggins,et al.  Methods for Combining Statistical Models of Music , 2004, CMMR.

[41]  M. Pearce,et al.  Sweet Anticipation : Music and the Psychology of Expectation , 2007 .

[42]  Eric Clarke,et al.  Ways of Listening: An Ecological Approach to the Perception of Musical Meaning , 2005 .

[43]  J. Martinerie,et al.  The brainweb: Phase synchronization and large-scale integration , 2001, Nature Reviews Neuroscience.

[44]  Stefan Koelsch,et al.  Bach Speaks: A Cortical “Language-Network” Serves the Processing of Music , 2002, NeuroImage.

[45]  B. Ross,et al.  Beta and Gamma Rhythms in Human Auditory Cortex during Musical Beat Processing , 2009, Annals of the New York Academy of Sciences.

[46]  G. Pfurtscheller,et al.  Event-related beta EEG-changes during passive and attempted foot movements in paraplegic patients , 2007, Brain Research.

[47]  D. Temperley The Cognition of Basic Musical Structures , 2001 .

[48]  Mayumi Adachi,et al.  Expectancy in melody: tests of children and adults. , 2002, Journal of experimental psychology. General.

[49]  I. Peretz Brain Specialization for Music , 2001, Annals of the New York Academy of Sciences.

[50]  H. Jasper Report of the committee on methods of clinical examination in electroencephalography , 1958 .

[51]  C. Stam,et al.  Phase lag index: Assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources , 2007, Human brain mapping.

[52]  Jürgen Kurths,et al.  Detection of n:m Phase Locking from Noisy Data: Application to Magnetoencephalography , 1998 .

[53]  Bret Aarden Dynamic melodic expectancy , 2003 .

[54]  Isabelle Peretz,et al.  Brain specialization for music. , 2002, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[55]  Geraint A. Wiggins,et al.  EXPECTATION IN MELODY: THE INFLUENCE OF CONTEXT AND LEARNING , 2006 .

[56]  Carol L Krumhansl,et al.  Experimental Strategies for Understanding the Role of Experience in Music Cognition , 2003, Annals of the New York Academy of Sciences.

[57]  Karim Jerbi,et al.  Simultaneous MEG and intracranial EEG recordings during attentive reading , 2009, NeuroImage.

[58]  Ian H. Witten,et al.  Text Compression , 1990, 125 Problems in Text Algorithms.

[59]  C. Krumhansl,et al.  Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys. , 1982 .

[60]  C. Krumhansl Cognitive Foundations of Musical Pitch , 1990 .

[61]  R Verleger,et al.  P3-evoking wrong notes: unexpected, awaited, or arousing? , 1990, The International journal of neuroscience.

[62]  C C Wood,et al.  Event-related potentials elicited by deviant endings to melodies. , 1992, Psychophysiology.

[63]  G. Pfurtscheller,et al.  On the existence of different types of central beta rhythms below 30 Hz. , 1997, Electroencephalography and clinical neurophysiology.

[64]  M. Whittington,et al.  Gamma and beta frequency oscillations in response to novel auditory stimuli: A comparison of human electroencephalogram (EEG) data with in vitro models. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[65]  Bhavin R. Sheth,et al.  Posterior Beta and Anterior Gamma Oscillations Predict Cognitive Insight , 2009, Journal of Cognitive Neuroscience.

[66]  O. Bertrand,et al.  Oscillatory gamma-band (30-70 Hz) activity induced by a visual search task in humans. , 1997, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[67]  Leonard B. Meyer Emotion and Meaning in Music , 1957 .

[68]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[69]  V. A. Makarov,et al.  Recovering EEG brain signals: Artifact suppression with wavelet enhanced independent component analysis , 2006, Journal of Neuroscience Methods.

[70]  S. Koelsch,et al.  Effects of Unexpected Chords and of Performer's Expression on Brain Responses and Electrodermal Activity , 2008, PloS one.

[71]  M. Besson,et al.  AN EVENT-RELATED POTENTIAL (ERP) STUDY OF MUSICAL EXPECTANCY : COMPARISON OF MUSICIANS WITH NONMUSICIANS , 1995 .

[72]  Schellenberg Eg Expectancy in melody: tests of the implication-realization model , 1996 .

[73]  J. Pernier,et al.  Oscillatory γ-Band (30–70 Hz) Activity Induced by a Visual Search Task in Humans , 1997, The Journal of Neuroscience.

[74]  John Hale,et al.  Uncertainty About the Rest of the Sentence , 2006, Cogn. Sci..

[75]  Jose M Hurtado,et al.  Statistical method for detection of phase-locking episodes in neural oscillations. , 2004, Journal of neurophysiology.

[76]  A. Schnitzler,et al.  Normal and pathological oscillatory communication in the brain , 2005, Nature Reviews Neuroscience.

[77]  Istvan Molnar-Szakacs,et al.  Music and mirror neurons: from motion to 'e'motion. , 2006, Social cognitive and affective neuroscience.

[78]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[79]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[80]  M. Woldorff,et al.  Effects of attention on the neural processing of harmonic syntax in Western music. , 2005, Brain research. Cognitive brain research.

[81]  Michael T. Ullman,et al.  Double dissociation between rules and memory in music: An event-related potential study , 2007, NeuroImage.

[82]  T. Gunter,et al.  Music matters: preattentive musicality of the human brain. , 2002, Psychophysiology.

[83]  M Kukleta,et al.  Beta 2-band synchronization during a visual oddball task. , 2009, Physiological research.

[84]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[85]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[86]  Isabelle Peretz,et al.  Book Review: Brain Specialization for Music , 2002 .

[87]  Suzanne Bunton,et al.  Semantically Motivated Improvements for PPM Variants , 1997, Comput. J..

[88]  J. Bharucha,et al.  Reaction time and musical expectancy: priming of chords. , 1986, Journal of experimental psychology. Human perception and performance.

[89]  J. Kelso,et al.  Cortical coordination dynamics and cognition , 2001, Trends in Cognitive Sciences.

[90]  P. Brown,et al.  Event-related beta desynchronization in human subthalamic nucleus correlates with motor performance. , 2004, Brain : a journal of neurology.

[91]  Isabelle Peretz,et al.  Toward a Biological Account of Music Experience , 2000, Brain and Cognition.

[92]  Stefan Koelsch,et al.  The Role of Harmonic Expectancy Violations in Musical Emotions: Evidence from Subjective, Physiological, and Neural Responses , 2006, Journal of Cognitive Neuroscience.

[93]  M. Kisley,et al.  Gamma and beta neural activity evoked during a sensory gating paradigm: Effects of auditory, somatosensory and cross-modal stimulation , 2006, Clinical Neurophysiology.