An investigation of annotation smoothing for EEG-based continuous music-emotion recognition

As emotional responses of a human to stimuli could evolve over the course of time, continuous emotion reporting is essential for the construction of a computational model to capture the temporal evolution of the human emotions. However, continuous emotion assessment is confronting various challenges, especially when using the continuous arousal-valence space. Manipulating emotion annotation data prior to performing emotion recognition is, therefore, necessary. In this paper, we present a study of applying three different signal filtering techniques to smooth annotation data; moving average filter, Savitzky-Golay filter, and median filter. We performed experiments of arousal and valence recognition in music listening tasks employing signals from electroencephalogram (EEG). Fractal dimension approach was adopted to extract informative features from brain dynamics and emotional states were then derived by classification and regression techniques. Our empirical results suggested the promise of the moving average filter that could enhance the performance of emotion classifying and tracking.

[1]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[2]  Olga Sourina,et al.  Real-time EEG-based emotion monitoring using stable features , 2015, The Visual Computer.

[3]  Masayuki Numao,et al.  Familiarity effects in EEG-based emotion recognition , 2016, Brain Informatics.

[4]  Roddy Cowie,et al.  FEELTRACE: an instrument for recording perceived emotion in real time , 2000 .

[5]  T. Higuchi Approach to an irregular time series on the basis of the fractal theory , 1988 .

[6]  Olga Sourina,et al.  Real-time EEG-based emotion recognition for music therapy , 2011, Journal on Multimodal User Interfaces.

[7]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[8]  Mohammad Soleymani,et al.  Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection , 2016, IEEE Transactions on Affective Computing.

[9]  Masayuki Numao,et al.  Continuous Music-Emotion Recognition Based on Electroencephalogram , 2016, IEICE Trans. Inf. Syst..

[10]  J. Russell A circumplex model of affect. , 1980 .

[11]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[12]  Hung T. Nguyen,et al.  Investigation of window size in classification of EEG-emotion signal with wavelet entropy and support vector machine , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Olga Sourina,et al.  Stability of Features in Real-Time EEG-based Emotion Recognition Algorithm , 2014, 2014 International Conference on Cyberworlds.

[14]  Björn W. Schuller,et al.  Categorical and dimensional affect analysis in continuous input: Current trends and future directions , 2013, Image Vis. Comput..

[15]  Carlos Busso,et al.  Analysis and Compensation of the Reaction Lag of Evaluators in Continuous Emotional Annotations , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[16]  T. Jung,et al.  Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening , 2014, Front. Neurosci..

[17]  Angeliki Metallinou,et al.  Annotation and processing of continuous emotional attributes: Challenges and opportunities , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[18]  B. Matthews Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[19]  Arnaud Delorme,et al.  EEGLAB, SIFT, NFT, BCILAB, and ERICA: New Tools for Advanced EEG Processing , 2011, Comput. Intell. Neurosci..

[20]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[21]  Maja Stikic,et al.  EEG-based classification of positive and negative affective states , 2014 .