Continuous Music-Emotion Recognition Based on Electroencephalogram

Research on emotion recognition using electroencephalogram (EEG) of subjects listening to music has become more active in the past decade. However, previous works did not consider emotional oscillations within a single musical piece. In this research, we propose a continuous music-emotion recognition approach based on brainwave signals. While considering the subject-dependent and changing-over-time characteristics of emotion, our experiment included self-reporting and continuous emotion annotation in the arousal-valence space. Fractal dimension (FD) and power spectral density (PSD) approaches were adopted to extract informative features from raw EEG signals and then we applied emotion classification algorithms to discriminate binary classes of emotion. According to our experimental results, FD slightly outperformed PSD approach both in arousal and valence classification, and FD was found to have the higher correlation with emotion reports than PSD. In addition, continuous emotion recognition during music listening based on EEG was found to be an effective method for tracking emotional reporting oscillations and provides an opportunity to better understand human emotional processes. key words: music, emotion, electroencephalogram

[1]  M. Grigutsch,et al.  Music and emotion: electrophysiological correlates of the processing of pleasant and unpleasant music. , 2007, Psychophysiology.

[2]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[3]  Guillaume Chanel,et al.  Emotion Assessment: Arousal Evaluation Using EEG's and Peripheral Physiological Signals , 2006, MRCS.

[4]  E. Altenmüller,et al.  Emotions over time: synchronicity and development of subjective, physiological, and facial affective reactions to music. , 2007, Emotion.

[5]  S. Koelsch.  Brain correlates of music-evoked emotions , 2014, Nature Reviews Neuroscience.

[6]  D. O. Bos,et al.  EEG-based Emotion Recognition The Influence of Visual and Auditory Stimuli , 2007 .

[7]  Omar AlZoubi,et al.  Classification of EEG for Affect Recognition: An Adaptive Approach , 2009, Australasian Conference on Artificial Intelligence.

[8]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[9]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition from Brain Signals Using Hybrid Adaptive Filtering and Higher Order Crossings Analysis , 2010, IEEE Transactions on Affective Computing.

[10]  B. Weiss,et al.  Comparison of fractal and power spectral EEG features: Effects of topography and sleep stages , 2011, Brain Research Bulletin.

[11]  T. Baumgartner,et al.  From emotion perception to emotion experience: emotions evoked by pictures and classical music. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[12]  A. Friederici,et al.  Investigating emotion with music: An fMRI study , 2006, Human brain mapping.

[13]  Olga Sourina,et al.  A Fractal-based Algorithm of Emotion Recognition from EEG using Arousal-Valence Model , 2011, BIOSIGNALS.

[14]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[15]  Olga Sourina,et al.  EEG Databases for Emotion Recognition , 2013, 2013 International Conference on Cyberworlds.

[16]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[17]  Kai Keng Ang,et al.  Affective computation on EEG correlates of emotion from musical and vocal stimuli , 2009, 2009 International Joint Conference on Neural Networks.

[18]  Duncan Williams,et al.  Affective Jukebox: A Confirmatory Study of EEG Emotional Correlates in Response to Musical Stimuli , 2014, ICMC.

[19]  Amit Konar,et al.  Emotion Recognition: A Pattern Analysis Approach , 2015 .

[20]  J. Russell.  A circumplex model of affect. , 1980 .

[21]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[22]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[23]  T. Higuchi.  Approach to an irregular time series on the basis of the fractal theory , 1988 .

[24]  Yi-Hsuan Yang,et al.  Machine Recognition of Music Emotion: A Review , 2012, TIST.

[25]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[26]  L. Trainor,et al.  Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions , 2001 .

[27]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[28]  Hiie Hinrikus,et al.  Spectral Asymmetry and Higuchi's Fractal Dimension Measures of Depression Electroencephalogram , 2013, Comput. Math. Methods Medicine.

[29]  Arnaud Delorme,et al.  EEGLAB, SIFT, NFT, BCILAB, and ERICA: New Tools for Advanced EEG Processing , 2011, Comput. Intell. Neurosci..

[30]  T. Jung,et al.  Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening , 2014, Front. Neurosci..

[31]  Bao-Liang Lu,et al.  Emotional state classification from EEG data using machine learning approach , 2014, Neurocomputing.

[32]  Olga Sourina,et al.  Real-time EEG-based emotion recognition for music therapy , 2011, Journal on Multimodal User Interfaces.

[33]  Yaacob Sazali,et al.  Classification of human emotion from EEG using discrete wavelet transform , 2010 .

[34]  S. A. Hosseini,et al.  Higher Order Spectra Analysis of EEG Signals in Emotional Stress States , 2010, 2010 Second International Conference on Information Technology and Computer Science.