Toward automatic detection of brain responses to emotional music through analysis of EEG effective connectivity

The purpose of this study was to investigate the effective brain networks associated with joyful, melancholic, and neutral music. Connectivity patterns among EEG electrodes in different frequency bands were extracted by multivariate autoregressive modeling while 19 nonmusicians listened to selected classical and Iranian musical excerpts. Musical selections were categorized according to the participants' average self-assessment results. Connectivity matrices were analyzed to identify distinct variations in the connectivity indices related to the categorized excerpts. We studied the correlation of inter-/intra-regional connectivity patterns with the self-reported evaluations of the musical selections. The perceived valence was positively correlated with the frontal inter-hemispheric flow, but negatively correlated with the parietal bilateral connectivity. Using the connectivity indices between different cortical areas and a support vector machine, we sought to distinguish trials in terms of the self-reported valence of perceived emotions and the familiarity of the musical genres. For 16 participants, the average classification accuracies in discriminating joyful from neutral, joyful from melancholic and familiar from unfamiliar trials were 93.7%???1.06%, 80.43%???1.74%, and 83.04%???1.47, respectively. Integration of different cortical areas is required for music perception and emotional processing. Thus, by studying the connectivity of brain regions, we may be able to develop a noninvasive assessment tool for investigating musical emotions. We used both classical and Iranian musical excerpts.We recorded EEG signals while participants listened to musical selections.We defined features based on the theory effective connectivity.We examined the correlation of the extracted features with valence and arousal.We classified signals into different categories using connectivity-based features.

[1]  Changle Zhou,et al.  Graph theoretical analysis of EEG functional connectivity during music perception , 2012, Brain Research.

[2]  R. Davidson.  Cerebral asymmetry, emotion, and affective style. , 1995 .

[3]  P. Vuilleumier,et al.  How brains beware: neural mechanisms of emotional attention , 2005, Trends in Cognitive Sciences.

[4]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[5]  Stefan Koelsch,et al.  Functional centrality of amygdala, striatum and hypothalamus in a “small‐world” network underlying joy: An fMRI study with music , 2014, Human brain mapping.

[6]  Katarzyna J. Blinowska,et al.  A new method of the description of the information flow in the brain structures , 1991, Biological Cybernetics.

[7]  M. Grigutsch,et al.  Music and emotion: electrophysiological correlates of the processing of pleasant and unpleasant music. , 2007, Psychophysiology.

[8]  H. Akaike.  A new look at the statistical model identification , 1974 .

[9]  A. Furnham,et al.  Trait emotional intelligence: behavioural validation in two studies of emotion recognition and reactivity to mood induction , 2003 .

[10]  C.W. Anderson,et al.  Comparison of linear, nonlinear, and feature selection methods for EEG signal classification , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  R. Zatorre,et al.  Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[13]  F. Babiloni,et al.  Estimation of the cortical functional connectivity with the multimodal integration of high-resolution EEG and fMRI data by directed transfer function , 2005, NeuroImage.

[14]  O. Sporns,et al.  Complex brain networks: graph theoretical analysis of structural and functional systems , 2009, Nature Reviews Neuroscience.

[15]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[16]  I. Peretz,et al.  Music and emotion: perceptual determinants, immediacy, and isolation after brain damage , 1998, Cognition.

[17]  Leontios J. Hadjileontiadis,et al.  Toward an EEG-Based Recognition of Music Liking Using Time-Frequency Analysis , 2012, IEEE Transactions on Biomedical Engineering.

[18]  Bin He,et al.  Electrophysiological Imaging of Brain Activity and Connectivity—Challenges and Opportunities , 2011, IEEE Transactions on Biomedical Engineering.

[19]  V. Makhnev,et al.  Neurophysiological Correlates of Induced Discrete Emotions in Humans: An Individually Oriented Analysis , 2006, Neuroscience and Behavioral Physiology.

[20]  Laura Astolfi,et al.  A new Kalman filter approach for the estimation of high-dimensional time-variant multivariate AR models and its application in analysis of laser-evoked brain potentials , 2010, NeuroImage.

[21]  R. E. Greenblatt,et al.  Connectivity measures applied to human brain electrophysiological data , 2012, Journal of Neuroscience Methods.

[22]  R. Zatorre,et al.  Anatomically distinct dopamine release during anticipation and experience of peak emotion to music , 2011, Nature Neuroscience.

[23]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[24]  J. Szaflarski,et al.  Moderating effects of music on resting state networks , 2012, Brain Research.

[25]  S. Koelsch.  Towards a neural basis of music-evoked emotions , 2010, Trends in Cognitive Sciences.

[26]  L. Schmidt,et al.  Cross-regional cortical synchronization during affective image viewing , 2010, Brain Research.

[27]  Roddy Cowie,et al.  FEELTRACE: an instrument for recording perceived emotion in real time , 2000 .

[28]  Zhen Ji,et al.  Gabor Wavelet Selection and SVM Classification for Object Recognition , 2009 .

[29]  Clemens Brunner,et al.  Single-trial connectivity estimation for classification of motor imagery data , 2013, Journal of neural engineering.

[30]  F. Barrios,et al.  Metabolic and electric brain patterns during pleasant and unpleasant emotions induced by music masterpieces. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[31]  Mingzhou Ding,et al.  Evaluating causal relations in neural systems: Granger causality, directed transfer function and statistical assessment of significance , 2001, Biological Cybernetics.

[32]  Leontios J. Hadjileontiadis,et al.  EEG-Based Classification of Music Appraisal Responses Using Time-Frequency Analysis and Familiarity Ratings , 2013, IEEE Transactions on Affective Computing.

[33]  Christof Karmonik,et al.  Graph theoretical connectivity analysis of the human brain while listening to music with emotional attachment: Feasibility study , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[34]  E. Altenmüller,et al.  Hits to the left, flops to the right: different emotions during listening to music are reflected in cortical lateralisation patterns , 2002, Neuropsychologia.

[35]  L. Trainor,et al.  Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions , 2001 .

[36]  Q. Wang,et al.  Real-Time Mental Arithmetic Task Recognition From EEG Signals , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  Claudio Babiloni,et al.  Functional frontoparietal connectivity during encoding and retrieval processes follows HERA model A high-resolution study , 2006, Brain Research Bulletin.

[38]  T. Baumgartner,et al.  From emotion perception to emotion experience: emotions evoked by pictures and classical music. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[39]  R. Davidson.  What does the prefrontal cortex “do” in affect: perspectives on frontal EEG asymmetry research , 2004, Biological Psychology.

[40]  A. Friederici,et al.  Investigating emotion with music: An fMRI study , 2006, Human brain mapping.

[41]  T. Chau,et al.  Automatic detection of a prefrontal cortical response to emotionally rated music using multi-channel near-infrared spectroscopy , 2012, Journal of neural engineering.

[42]  M. Balconi,et al.  Consciousness and arousal effects on emotional face processing as revealed by brain oscillations. A gamma band analysis. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[43]  Bin He,et al.  Estimation of Time-Varying Connectivity Patterns Through the Use of an Adaptive Directed Transfer Function , 2008, IEEE Transactions on Biomedical Engineering.

[44]  Kamryn T. Eddy,et al.  Amygdala-frontal connectivity during emotion regulation. , 2007, Social cognitive and affective neuroscience.

[45]  Leontios J. Hadjileontiadis,et al.  A Novel Emotion Elicitation Index Using Frontal Brain Asymmetry for Enhanced EEG-Based Emotion Recognition , 2011, IEEE Transactions on Information Technology in Biomedicine.

[46]  D. Västfjäll,et al.  Emotional responses to music: the need to consider underlying mechanisms. , 2008, The Behavioral and brain sciences.

[47]  Stefan Koelsch,et al.  Brain and Music , 2012 .

[48]  Rosalind W. Picard.  Affective Computing: From Laughter to IEEE , 2010 .

[49]  Joydeep Bhattacharya,et al.  Phase synchrony analysis of EEG during music perception reveals changes in functional connectivity due to musical expertise , 2005, Signal Process..