Multi-Variate EEG Analysis as a Novel Tool to Examine Brain Responses to Naturalistic Music Stimuli

Note onsets in music are acoustic landmarks providing auditory cues that underlie the perception of more complex phenomena such as beat, rhythm, and meter. For naturalistic ongoing sounds a detailed view on the neural representation of onset structure is hard to obtain, since, typically, stimulus-related EEG signatures are derived by averaging a high number of identical stimulus presentations. Here, we propose a novel multivariate regression-based method extracting onset-related brain responses from the ongoing EEG. We analyse EEG recordings of nine subjects who passively listened to stimuli from various sound categories encompassing simple tone sequences, full-length romantic piano pieces and natural (non-music) soundscapes. The regression approach reduces the 61-channel EEG to one time course optimally reflecting note onsets. The neural signatures derived by this procedure indeed resemble canonical onset-related ERPs, such as the N1-P2 complex. This EEG projection was then utilized to determine the Cortico-Acoustic Correlation (CACor), a measure of synchronization between EEG signal and stimulus. We demonstrate that a significant CACor (i) can be detected in an individual listener's EEG of a single presentation of a full-length complex naturalistic music stimulus, and (ii) it co-varies with the stimuli’s average magnitudes of sharpness, spectral centroid, and rhythmic complexity. In particular, the subset of stimuli eliciting a strong CACor also produces strongly coordinated tension ratings obtained from an independent listener group in a separate behavioral experiment. Thus musical features that lead to a marked physiological reflection of tone onsets also contribute to perceived tension in music.

[1]  Lutz Jäncke,et al.  Electrical brain imaging reveals spatio-temporal dynamics of timbre perception in humans , 2006, NeuroImage.

[2]  Antoine J. Shahin,et al.  Enhancement of Neuroplastic P2 and N1c Auditory Evoked Potentials in Musicians , 2003, The Journal of Neuroscience.

[3]  James Theiler,et al.  Testing for nonlinearity in time series: the method of surrogate data , 1992 .

[4]  Edgard Varèse,et al.  The Liberation of Sound , 1966 .

[5]  Pedro R. Almeida,et al.  Effects of inter-stimulus interval (ISI) duration on the N1 and P2 components of the auditory event-related potential. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[6]  Christoph Mulert,et al.  Sound level dependence of the primary auditory cortex: Simultaneous measurement with 61-channel EEG and fMRI , 2005, NeuroImage.

[7]  J. Sloboda,et al.  Handbook of Music and Emotion: Theory, Research, Applications , 2011 .

[8]  F. Graeff,et al.  Reciprocal Modulation of Cognitive and Emotional Aspects in Pianistic Performances , 2011, PloS one.

[9]  Hermann Ackermann,et al.  Magnetic brain activity phase-locked to the envelope, the syllable onsets, and the fundamental frequency of a perceived speech signal. , 2012, Psychophysiology.

[10]  Jason Farquhar,et al.  Name that tune: Decoding music from the listening brain , 2011, NeuroImage.

[11]  R. Peterman,et al.  Comparison of methods to account for autocorrelation in correlation analyses of fish data , 1998 .

[12]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[13]  J. Simon,et al.  Cortical entrainment to continuous speech: functional roles and interpretations , 2014, Front. Hum. Neurosci..

[14]  J. Russell A circumplex model of affect. , 1980 .

[15]  M. Farbood,et al.  Determining Feature Relevance in Subject Responses to Musical Stimuli , 2009 .

[16]  Ray Jackendoff,et al.  An overview of hierarchical structure in music , 1983 .

[17]  Antoine J. Shahin,et al.  Modulation of P2 auditory-evoked responses by the spectral complexity of musical sounds , 2005, Neuroreport.

[18]  W. Thompson,et al.  A Comparison of Acoustic Cues in Music and Speech for Three Dimensions of Affect , 2006 .

[19]  Klaus-Robert Müller,et al.  Temporal kernel CCA and its application in multimodal neuronal data analysis , 2010, Machine Learning.

[20]  Richard J. Davidson,et al.  Electromyogenic artifacts and electroencephalographic inferences revisited , 2011, NeuroImage.

[21]  M. Farbood,et al.  Interpreting expressive performance through listener judgments of musical tension , 2013, Front. Psychol..

[22]  John J. Foxe,et al.  Resolving precise temporal processing properties of the auditory system using continuous stimuli. , 2009, Journal of neurophysiology.

[23]  S. Hillyard,et al.  Electrical Signs of Selective Attention in the Human Brain , 1973, Science.

[24]  Mikko Sams,et al.  Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm , 2012, NeuroImage.

[25]  Anders Askenfelt Speech , Music and Hearing Quarterly Progress and Status Report Observations on the transient components of the piano tone , 2007 .

[26]  J.C. Mosher,et al.  Recursive MUSIC: A framework for EEG and MEG source localization , 1998, IEEE Transactions on Biomedical Engineering.

[27]  José Fornari,et al.  Multi-Feature Modeling of Pulse Clarity: Design, Validation and Optimization , 2008, ISMIR.

[28]  Emery Schubert,et al.  Continuous Self-Report Methods , 1993 .

[29]  Antoine J. Shahin,et al.  Development of auditory phase-locked activity for music sounds. , 2010, Journal of neurophysiology.

[30]  David Poeppel,et al.  Acoustic landmarks drive delta–theta oscillations to enable speech comprehension by facilitating perceptual parsing , 2014, NeuroImage.

[31]  T. Picton,et al.  The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. , 1987, Psychophysiology.

[32]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[33]  Tapani Ristaniemi,et al.  Analysis of ongoing EEG elicited by natural music stimuli using nonnegative tensor factorization , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[34]  Petri Toiviainen,et al.  A Matlab Toolbox for Music Information Retrieval , 2007, GfKl.

[35]  John J. Foxe,et al.  Neural responses to uninterrupted natural speech can be extracted with precise temporal resolution , 2010, The European journal of neuroscience.

[36]  S. Koelsch,et al.  The Influence of Different Structural Features on Felt Musical Tension in Two Piano Pieces by Mozart and Mendelssohn , 2013 .

[37]  Lutz Jäncke,et al.  Time course of EEG oscillations during repeated listening of a well-known aria , 2015, Front. Hum. Neurosci..

[38]  S. Koelsch,et al.  Predictive information processing in music cognition. A critical review. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[39]  M. Farbood A Parametric, Temporal Model of Musical Tension , 2012 .

[40]  B. Ross,et al.  Is the auditory evoked P2 response a biomarker of learning? , 2014, Front. Syst. Neurosci..

[41]  Terence W Picton,et al.  Human temporal auditory acuity as assessed by envelope following responses. , 2004, The Journal of the Acoustical Society of America.

[42]  Maurizio Vichi,et al.  Studies in Classification Data Analysis and knowledge Organization , 2011 .

[43]  M. Zentner,et al.  Homer's Prophecy: an Essay on Music's Primary Emotions , 2010 .

[44]  R. Kakigi,et al.  One year of musical training affects development of auditory cortical-evoked fields in young children. , 2006, Brain : a journal of neurology.

[45]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[46]  S. Koelsch,et al.  Toward a general psychological model of tension and suspense , 2015, Front. Psychol..

[47]  A. Norcia,et al.  TOWARD AN OBJECTIVE MEASURE OF LISTENER ENGAGEMENT WITH NATURAL MUSIC USING INTER-SUBJECT EEG CORRELATION , 2014 .

[48]  T. Picton,et al.  Human Cortical Responses to the Speech Envelope , 2008, Ear and hearing.

[49]  Birgitta Burger,et al.  Influences of Rhythm- and Timbre-Related Musical Features on Characteristics of Music-Induced Movement , 2013, Front. Psychol..

[50]  C. Krumhansl A Perceptual Analysis of Mozart's Piano Sonata K. 282: Segmentation, Tension, and Musical Ideas , 1996 .

[51]  Terence W. Picton,et al.  Envelope Following Responses to Natural Vowels , 2006, Audiology and Neurotology.

[52]  Petri Toiviainen,et al.  Prediction of Multidimensional Emotional Ratings in Music from Audio Using Multivariate Regression Models , 2009, ISMIR.

[53]  F. I. Corona-Strauss,et al.  Extraction of Auditory Attention Correlates in Single Sweeps of Cortical Potentials by Maximum Entropy Paradigms and its Application , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[54]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[55]  Antoine J. Shahin,et al.  Attentional Gain Control of Ongoing Cortical Speech Representations in a “Cocktail Party” , 2010, The Journal of Neuroscience.

[56]  Lucas C. Parra,et al.  Recipes for the linear analysis of EEG , 2005, NeuroImage.

[57]  Maurice G. Kendall,et al.  Inference and relationship , 1979 .

[58]  G. Curio,et al.  ECoG high gamma activity reveals distinct cortical representations of lyrics passages, harmonic and timbre-related changes in a rock song , 2014, Front. Hum. Neurosci..

[59]  Andreas Ziehe,et al.  A Fast Algorithm for Joint Diagonalization with Non-orthogonal Transformations and its Application to Blind Source Separation , 2004, J. Mach. Learn. Res..

[60]  Morwaread Farbood,et al.  A quantitative, parametric model of musical tension , 2006 .

[61]  Ingo Fründ,et al.  Stimulus intensity affects early sensory processing: sound intensity modulates auditory evoked gamma-band activity in human EEG. , 2007, International Journal of Psychophysiology.

[62]  Patrick Suppes,et al.  Structural decomposition of EEG signatures of melodic processing , 2009, Biological Psychology.

[63]  Z. Eitan,et al.  Musical Tension and the Interaction of Dynamic Auditory Parameters , 2011 .

[64]  Klaus-Robert Müller,et al.  Stereoscopic depth increases intersubject correlations of brain networks , 2014, NeuroImage.

[65]  Limits on the Application of Statistical Correlations to Continuous Response Data , 2012 .

[66]  R. Turner,et al.  Deficient approaches to human neuroimaging , 2014, Front. Hum. Neurosci..

[67]  Shigeyuki Kuwada,et al.  Scalp potentials of normal and hearing-impaired subjects in response to sinusoidally amplitude-modulated tones , 1986, Hearing Research.

[68]  Jessie L. Thompson Neural decoding of subjective music listening experiences , 2013 .

[69]  G. Madison,et al.  Modeling the tendency for music to induce movement in humans: first correlations with low-level audio descriptors across music genres. , 2011, Journal of experimental psychology. Human perception and performance.

[70]  Richard J. Davidson,et al.  Validation of ICA-based myogenic artifact correction for scalp and source-localized EEG , 2010, NeuroImage.