EEG-Based Emotion Classification Using Spiking Neural Networks

A novel method of using the spiking neural networks (SNNs) and the electroencephalograph (EEG) processing techniques to recognize emotion states is proposed in this paper. Three algorithms including discrete wavelet transform (DWT), variance and fast Fourier transform (FFT) are employed to extract the EEG signals, which are further taken by the SNN for the emotion classification. Two datasets, i.e., DEAP and SEED, are used to validate the proposed method. For the former dataset, the emotional states include arousal, valence, dominance and liking where each state is denoted as either high or low status. For the latter dataset, the emotional states are divided into three categories (negative, positive and neutral). Experimental results show that by using the variance data processing technique and SNN, the emotion states of arousal, valence, dominance and liking can be classified with accuracies of 74%, 78%, 80% and 86.27% for the DEAP dataset, and an overall accuracy is 96.67% for the SEED dataset, which outperform the FFT and DWT processing methods. In the meantime, this work achieves a better emotion classification performance than the benchmarking approaches, and also demonstrates the advantages of using SNN for the emotion state classifications.

[1]  Tiago H. Falk,et al.  Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization , 2016, Neurocomputing.

[2]  Susan P. Worner,et al.  NeuCube(ST) for spatio-temporal data predictive modelling with a case study on ecological data , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[3]  Goutam Saha,et al.  Classification of emotions induced by music videos and correlation with participants' rating , 2014, Expert Syst. Appl..

[4]  Su Yang,et al.  Echo state network optimization using binary grey wolf algorithm , 2020, Neurocomputing.

[5]  Nikola Kasabov,et al.  Feasibility of NeuCube spiking neural network architecture for EMG pattern recognition , 2015, 2015 International Conference on Advanced Mechatronic Systems (ICAMechS).

[6]  K. H. Kim,et al.  Emotion recognition system using short-term monitoring of physiological signals , 2004, Medical and Biological Engineering and Computing.

[7]  Tobi Delbrück,et al.  Fast sensory motor control based on event-based hybrid neuromorphic-procedural system , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[8]  Wenwei Yu,et al.  EMG prosthetic hand controller using real-time learning method , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[9]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[10]  Yaacob Sazali,et al.  Classification of human emotion from EEG using discrete wavelet transform , 2010 .

[11]  Huosheng Hu,et al.  Support Vector Machine-Based Classification Scheme for Myoelectric Control Applied to Upper Limb , 2008, IEEE Transactions on Biomedical Engineering.

[12]  Jin Hu,et al.  Feasibility of NeuCube SNN architecture for detecting motor execution and motor intention for use in BCIapplications , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[13]  Jim Harkin,et al.  Exploring Self-Repair in a Coupled Spiking Astrocyte Neural Network , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Liam McDaid,et al.  Bio-inspired fault detection circuits based on synapse and spiking neuron models , 2019, Neurocomputing.

[15]  Muhaini Othman,et al.  Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke , 2014, Neurocomputing.

[16]  Yi Cao,et al.  Low Cost Interconnected Architecture for the Hardware Spiking Neural Networks , 2018, Front. Neurosci..

[17]  Su Yang,et al.  An echo state network architecture based on quantum logic gate and its optimization , 2020, Neurocomputing.

[18]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[19]  Q. M. Jonathan Wu,et al.  EEG-Based Emotion Recognition Using Hierarchical Network With Subnetwork Nodes , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[20]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[21]  Héctor M. Pérez Meana,et al.  Facial expression recognition with automatic segmentation of face regions using a fuzzy based classification approach , 2016, Knowl. Based Syst..

[22]  Jim Harkin,et al.  SPANNER: A Self-Repairing Spiking Neural Network Hardware Architecture , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Bao-Liang Lu,et al.  Emotional state classification from EEG data using machine learning approach , 2014, Neurocomputing.

[24]  M. Shamim Hossain,et al.  An Emotion Recognition System for Mobile Applications , 2017, IEEE Access.

[25]  Adrian D. C. Chan,et al.  Continuous myoelectric control for powered prostheses using hidden Markov models , 2005, IEEE Transactions on Biomedical Engineering.

[26]  Hongtu Zhu,et al.  An affective circumplex model of neural systems subserving valence, arousal, and cognitive overlay during the appraisal of emotional faces , 2008, Neuropsychologia.

[27]  Kamlesh Mistry,et al.  Intelligent facial emotion recognition using moth-firefly optimization , 2016, Knowl. Based Syst..

[28]  Wenming Zheng,et al.  EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks , 2020, IEEE Transactions on Affective Computing.

[29]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[30]  Thierry Pun,et al.  Multimodal Emotion Recognition in Response to Videos , 2012, IEEE Transactions on Affective Computing.

[31]  Nikola Kasabov,et al.  Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. , 2013, Neural networks : the official journal of the International Neural Network Society.

[32]  Bao-Liang Lu,et al.  Personalizing EEG-Based Affective Models with Transfer Learning , 2016, IJCAI.

[33]  Giovanni Costantini,et al.  Speech emotion recognition using amplitude modulation parameters and a combined feature selection procedure , 2014, Knowl. Based Syst..

[34]  Jinling Wang,et al.  Mass Spectral Substance Detections Using Long Short-Term Memory Networks , 2019, IEEE Access.

[35]  Maryam Gholami Doborjeh,et al.  Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications , 2016, Neural Networks.

[36]  Olga Sourina,et al.  EEG-based Dominance Level Recognition for Emotion-Enabled Interaction , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[37]  Olga Sourina,et al.  Real-Time EEG-Based Human Emotion Recognition and Visualization , 2010, 2010 International Conference on Cyberworlds.

[38]  Wulfram Gerstner,et al.  What is Different with Spiking Neurons , 2001 .

[39]  John Atkinson,et al.  Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers , 2016, Expert Syst. Appl..

[40]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[41]  G. Saha,et al.  Recognition of emotions induced by music videos using DT-CWPT , 2013, 2013 Indian Conference on Medical Informatics and Telemedicine (ICMIT).

[42]  Saeid Nahavandi,et al.  EEG signal analysis for BCI application using fuzzy system , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[43]  Mohammad Soleymani,et al.  Single Trial Classification of EEG and Peripheral Physiological Signals for Recognition of Emotions Induced by Music Videos , 2010, Brain Informatics.

[44]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.