Automated emotion classification in the early stages of cortical processing: An MEG study

PURPOSE Here we aimed to automatically classify human emotion earlier than is typically attempted. There is increasing evidence that the human brain differentiates emotional categories within 100-300 ms after stimulus onset. Therefore, here we evaluate the possibility of automatically classifying human emotions within the first 300 ms after the stimulus and identify the time-interval of the highest classification performance. METHODS To address this issue, MEG signals of 17 healthy volunteers were recorded in response to three different picture stimuli (pleasant, unpleasant, and neutral pictures). Six Linear Discriminant Analysis (LDA) classifiers were used based on two binary comparisons (pleasant versus neutral and unpleasant versus neutral) and three different time-intervals (100-150 ms, 150-200 ms, and 200-300 ms post-stimulus). The selection of the feature subsets was performed by Genetic Algorithm and LDA. RESULTS We demonstrated significant classification performances in both comparisons. The best classification performance was achieved with a median AUC of 0.83 (95 %- CI [0.71; 0.87]) classifying brain responses evoked by unpleasant and neutral stimuli within 100-150 ms, which is at least 850 ms earlier than attempted by other studies. CONCLUSION Our results indicate that using the proposed algorithm, brain emotional responses can be significantly classified at very early stages of cortical processing (within 300 ms). Moreover, our results suggest that emotional processing in the human brain occurs within the first 100-150 ms.

[1]  James N. Knight,et al.  Feature Selection and Blind Source Separation in an EEG-Based Brain-Computer Interface , 2005, EURASIP J. Adv. Signal Process..

[2]  Bernhard Graimann,et al.  Toward a direct brain interface based on human subdural recordings and wavelet-packet analysis , 2004, IEEE Transactions on Biomedical Engineering.

[3]  Radoslaw Martin Cichy,et al.  Multivariate pattern analysis for MEG: A comparison of dissimilarity measures , 2018, NeuroImage.

[4]  Byoung-Jun Park,et al.  Classification of Human Emotions from Physiological signals using Machine Learning Algorithms , 2013, ACHI 2013.

[5]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[6]  Saleh A. Alshebeili,et al.  Epileptic MEG Spikes Detection Using Common Spatial Patterns and Linear Discriminant Analysis , 2016, IEEE Access.

[7]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[8]  Matthias M. Müller,et al.  Effects of emotional arousal in the cerebral hemispheres: a study of oscillatory brain activity and event-related potentials , 2001, Clinical Neurophysiology.

[9]  Martin Eimer,et al.  Rapid Detection of Emotion from Human Vocalizations , 2010, Journal of Cognitive Neuroscience.

[10]  Vanessa Sluming,et al.  Regional brain responses to pleasant and unpleasant IAPS pictures: Different networks , 2012, Neuroscience Letters.

[11]  Bao-Liang Lu,et al.  Emotion classification based on gamma-band EEG , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Sonja A. Kotz,et al.  Investigating the multimodal nature of human communication: Insights from ERPs. , 2009 .

[13]  Zhi-Long Chen,et al.  Using EEG to detect drivers' emotion with Bayesian Networks , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[14]  G.F. Inbar,et al.  Feature selection for the classification of movements from single movement-related potentials , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  João P. Cabral,et al.  MILLA Multimodal Interactive Language Learning Agent , 2014 .

[16]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[17]  Mohammad Hassan Moradi,et al.  A new approach for EEG feature extraction in P300-based lie detection , 2009, Comput. Methods Programs Biomed..

[18]  L. Obler,et al.  Right hemisphere emotional perception: evidence across multiple channels. , 1998, Neuropsychology.

[19]  Mohammad Bagher Shamsollahi,et al.  Selection of Efficient Features for Discrimination of Hand Movements from MEG Using a BCI Competition IV Data Set , 2012, Front. Neurosci..

[20]  R. Adolphs Neural systems for recognizing emotion , 2002, Current Opinion in Neurobiology.

[21]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[22]  Donatella Spinelli,et al.  Neural correlates of fast stimulus discrimination and response selection in top-level fencers , 2006, Neuroscience Letters.

[23]  Sonja A. Kotz,et al.  Selective Attention Modulates Early Human Evoked Potentials during Emotional Face–Voice Processing , 2015, Journal of Cognitive Neuroscience.

[24]  Bao-Liang Lu,et al.  EEG-based emotion recognition during watching movies , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[25]  C.W. Anderson,et al.  Comparison of linear, nonlinear, and feature selection methods for EEG signal classification , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Stephen D. Smith,et al.  The effects of valence and arousal on the emotional modulation of time perception: evidence for multiple stages of processing. , 2011, Emotion.

[27]  Tanja Schultz,et al.  Towards emotion recognition from electroencephalographic signals , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[28]  Roger B. H. Tootell,et al.  The advantage of combining MEG and EEG: Comparison to fMRI in focally stimulated visual cortex , 2007, NeuroImage.

[29]  Jin-Kao Hao,et al.  A hybrid LDA and genetic algorithm for gene selection and classification of microarray data , 2010, Neurocomputing.

[30]  Michela Balconi,et al.  Hemodynamic (fNIRS) and EEG (N200) correlates of emotional inter-species interactions modulated by visual and auditory stimulation , 2016, Scientific Reports.

[31]  Subramanian Ramanathan,et al.  DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses , 2015, IEEE Transactions on Affective Computing.

[32]  Kazuhiko Takahashi Remarks on Emotion Recognition from Bio-Potential Signals , 2004 .

[33]  H. Nordby,et al.  Event-related potential (ERP) asymmetries to emotional stimuli in a visual half-field paradigm. , 1997, Psychophysiology.

[34]  M. Codispoti,et al.  When does size not matter? Effects of stimulus size on affective modulation. , 2006, Psychophysiology.

[35]  Guillaume Chanel,et al.  Emotion Assessment: Arousal Evaluation Using EEG's and Peripheral Physiological Signals , 2006, MRCS.

[36]  D. Lehmann,et al.  Rapid emotional face processing in the human right and left brain hemispheres: an ERP study. , 1999, Neuroreport.

[37]  Dandan Huang,et al.  Single trial detection of human movement intentions from SAM-filtered MEG signals for a high performance two-dimensional BCI , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[38]  R. Dolan,et al.  Neural Activation during Covert Processing of Positive Emotional Facial Expressions , 1996, NeuroImage.

[39]  R. Kötter,et al.  Functional dissociation between medial and lateral prefrontal cortical spatiotemporal activation in negative and positive emotions: a combined fMRI/MEG study. , 2000, Cerebral cortex.

[40]  A. Ohman,et al.  Emotional conditioning to masked stimuli: expectancies for aversive outcomes following nonrecognized fear-relevant stimuli. , 1998, Journal of experimental psychology. General.

[41]  Pasin Israsena,et al.  EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation , 2014, TheScientificWorldJournal.

[42]  A. Damasio,et al.  Subcortical and cortical brain activity during the feeling of self-generated emotions , 2000, Nature Neuroscience.

[43]  Stefan Brodoehl,et al.  The Temporal and Spatial Dynamics of Cortical Emotion Processing in Different Brain Frequencies as Assessed Using the Cluster-Based Permutation Test: An MEG Study , 2020, Brain sciences.

[44]  Patrik Vuilleumier,et al.  Time course and specificity of event-related potentials to emotional expressions , 2004, Neuroreport.

[45]  P. Lang International Affective Picture System (IAPS) : Technical Manual and Affective Ratings , 1995 .

[46]  O. Witte,et al.  Abnormal Emotional Processing and Emotional Experience in Patients with Peripheral Facial Nerve Paralysis: An MEG Study , 2020, Brain sciences.

[47]  M. Bradley,et al.  Emotional arousal and activation of the visual cortex: an fMRI analysis. , 1998, Psychophysiology.

[48]  L. Darrell Whitley,et al.  Genetic Approach to Feature Selection for Ensemble Creation , 1999, GECCO.

[49]  S. Taulu,et al.  Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements , 2006, Physics in medicine and biology.

[50]  M. Bradley,et al.  Large-scale neural correlates of affective picture processing. , 2002, Psychophysiology.

[51]  Ana L. N. Fred,et al.  Unveiling the Biometric Potential of Finger-Based ECG Signals , 2011, Comput. Intell. Neurosci..

[52]  M Esslen,et al.  Brain areas and time course of emotional processing , 2004, NeuroImage.

[53]  Jin-Hun Sohn,et al.  Eeg-based emotion recogntion during emotionally evocative films , 2013, 2013 International Winter Workshop on Brain-Computer Interface (BCI).

[54]  Hanna Damasio,et al.  Single-neuron responses to emotional visual stimuli recorded in human ventral prefrontal cortex , 2001, Nature Neuroscience.

[55]  M. Junghöfer,et al.  The selective processing of briefly presented affective pictures: an ERP analysis. , 2004, Psychophysiology.

[56]  Saurabh Prasad,et al.  Genetic algorithms and Linear Discriminant Analysis based dimensionality reduction for remotely sensed image analysis , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[57]  Marcos Dipinto,et al.  Discriminant analysis , 2020, Predictive Analytics.

[58]  E. Halgren,et al.  Cognitive response profile of the human fusiform face area as determined by MEG. , 2000, Cerebral cortex.

[59]  Michel Besserve,et al.  Classification methods for ongoing EEG and MEG signals. , 2007, Biological research.