A Web Application for Characterizing Spontaneous Emotions Using Long EEG Recording Sessions

Emotions are important in daily life and in several research fields, especially in Brain Computer Interface (BCI) and Affective Computing. Usually, emotions are studied by analyzing the brain activity of a subject, monitored by Electroencephalography (EEG), functional Magnetic Resonance Imaging (fMRI) or functional Near Infrared Spectroscopy (fNIRS), after some external stimulation. This approach could lead to characterization inaccuracies, due to the secondary activations produced by the artificial elicitation and to the subjective emotional response. In this work, we design a web application to support spontaneous emotions characterization. It is based on a database for EEG signals where a large amount of data from long recording sessions, collected from subjects during their daily life, are stored. In this way, EEG signals can be explored to characterize different spontaneous emotional states felt by several people. The application is also designed to extract features of specific emotions, and to compare different emotional states. Researchers all over the world could share both raw data and classification results. Since large datasets are treated, the application is based on strategies commonly used in big data managing. In particular, a column-oriented database is used to store a huge amount of raw EEG signals, while a relational database is employed to keep metadata information. A web application interface allows the user to communicate with the repository and a computational module performs the features extraction.

[1]  C. W. Hughes.  Emotion: Theory, Research and Experience , 1982 .

[2]  Bob Kemp,et al.  European data format ‘plus’ (EDF+), an EDF alike standard format for the exchange of physiological data , 2003, Clinical Neurophysiology.

[3]  R. Henkin,et al.  Lateralization of Brain Activation to Imagination and Smell of Odors Using Functional Magnetic Resonance Imaging (fMRI): Left Hemispheric Localization of Pleasant and Right Hemispheric Localization of Unpleasant Odors , 2001, Journal of computer assisted tomography.

[4]  Valer Jurcak,et al.  10/20, 10/10, and 10/5 systems revisited: Their validity as relative head-surface-based positioning systems , 2007, NeuroImage.

[5]  Sara Basso Moro,et al.  A novel semi-immersive virtual reality visuo-motor task activates ventrolateral prefrontal cortex: a functional near-infrared spectroscopy study , 2016, Journal of neural engineering.

[6]  Rosalind W. Picard.  Affective Computing , 1997 .

[7]  M. Just,et al.  Identifying Emotions on the Basis of Neural Activation , 2013, PloS one.

[8]  Valentina Quaresima,et al.  Functional near-infrared frontal cortex imaging for virtual reality neuro-rehabilitation assessment , 2012 .

[9]  Sam Ruby,et al.  RESTful Web Services , 2007 .

[10]  Luigi Cinque,et al.  Time-of-Flight Camera Based Virtual Reality Interaction for Balance Rehabilitation Purposes , 2014, CompIMAGE.

[11]  N. Draper,et al.  Applied Regression Analysis , 1967 .

[12]  Andrea Petracca,et al.  A Modular Framework for EEG Web Based Binary Brain Computer Interfaces to Recover Communication Abilities in Impaired People , 2015, Journal of Medical Systems.

[13]  Giuseppe Placidi,et al.  A smart virtual glove for the hand telerehabilitation , 2007, Comput. Biol. Medicine.

[14]  Guan Le,et al.  Survey on NoSQL database , 2011, 2011 6th International Conference on Pervasive Computing and Applications.

[15]  Avraham Leff,et al.  Web-application development using the Model/View/Controller design pattern , 2001, Proceedings Fifth IEEE International Enterprise Distributed Object Computing Conference.

[16]  Anton Nijholt,et al.  Emotional brain-computer interfaces , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[17]  John D E Gabrieli,et al.  Control over brain activation and pain learned by using real-time functional MRI. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Danilo Avola,et al.  Design of an efficient framework for fast prototyping of customized human-computer interfaces and virtual environments for rehabilitation , 2013, Comput. Methods Programs Biomed..

[19]  Bao-Liang Lu,et al.  Emotional state classification from EEG data using machine learning approach , 2014, Neurocomputing.

[20]  Niels Birbaumer,et al.  Differences in hemodynamic activations between motor imagery and upper limb FES with NIRS , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[21]  Giuseppe Placidi,et al.  A numerical hand model for a virtual glove rehabilitation system , 2009, 2009 IEEE International Workshop on Medical Measurements and Applications.

[22]  Andrea Petracca,et al.  A Poll Oriented Classifier for Affective Brain Computer Interfaces , 2015, NEUROTECHNIX.

[23]  Stefan Jablonski,et al.  NoSQL evaluation: A use case oriented survey , 2011, 2011 International Conference on Cloud and Service Computing.

[24]  V. Quaresima,et al.  A virtual ball task driven by forearm movements for neuro-rehabilitation , 2015, 2015 International Conference on Virtual Rehabilitation (ICVR).

[25]  J. Huggins,et al.  Brain-computer interface: current and emerging rehabilitation applications. , 2015, Archives of physical medicine and rehabilitation.

[26]  Tilo Kircher,et al.  Acquired self‐control of insula cortex modulates emotion recognition and brain network connectivity in schizophrenia , 2013, Human brain mapping.

[27]  Andrea Petracca,et al.  A Classification Algorithm for Electroencephalography Signals by Self-Induced Emotional Stimuli , 2016, IEEE Transactions on Cybernetics.

[28]  Andrea Petracca,et al.  A real-time classification algorithm for EEG-based BCI driven by self-induced emotions , 2015, Comput. Methods Programs Biomed..

[29]  M. Balconi,et al.  Consciousness and arousal effects on emotional face processing as revealed by brain oscillations. A gamma band analysis. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[30]  Sanjay Ghemawat,et al.  MapReduce: a flexible data processing tool , 2010, CACM.

[31]  R. Plutchik,et al.  Emotion: Theory, Research, and Experience. Vol. 1. Theories of Emotion , 1981 .

[32]  Andrea Petracca,et al.  EEG-detected olfactory imagery to reveal covert consciousness in minimally conscious state , 2015, Brain injury.

[33]  Andrea Petracca,et al.  Classification strategies for a single-trial binary Brain Computer Interface based on remembering unpleasant odors , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[34]  Andrea Petracca,et al.  Classification of Emotional Signals from the DEAP dataset , 2016, NEUROTECHNIX.

[35]  Bao-Liang Lu,et al.  Emotion classification based on gamma-band EEG , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[36]  Cuntai Guan,et al.  Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface , 2007, NeuroImage.

[37]  J. Wolpaw.  Brain-computer interfaces. , 2013, Handbook of clinical neurology.

[38]  Huan Liu,et al.  Big Data Drives Cloud Adoption in Enterprise , 2013, IEEE Internet Computing.

[39]  Luigi Cinque,et al.  Markerless Hand Gesture Interface Based on LEAP Motion Controller , 2014, DMS.

[40]  Chang-Hwan Im,et al.  EEG-Based Brain-Computer Interfaces: A Thorough Literature Survey , 2013, Int. J. Hum. Comput. Interact..

[41]  Roy T. Fielding,et al.  Principled design of the modern Web architecture , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[42]  Domenico Talia,et al.  Clouds for Scalable Big Data Analytics , 2013, Computer.

[43]  Luigi Cinque,et al.  Towards EEG-based BCI driven by emotions for addressing BCI-Illiteracy: a meta-analytic review , 2018, Behav. Inf. Technol..

[44]  Divyakant Agrawal,et al.  Big data and cloud computing: current state and future opportunities , 2011, EDBT/ICDT '11.

[45]  L. Mazzola,et al.  Long-term EEG in adults: Sleep-deprived EEG (SDE), ambulatory EEG (Amb-EEG) and long-term video-EEG recording (LTVER) , 2015, Neurophysiologie Clinique/Clinical Neurophysiology.

[46]  Luigi Cinque,et al.  An Affective BCI Driven by Self-induced Emotions for People with Severe Neurological Disorders , 2017, ICIAP Workshops.

[47]  Ethan Marcotte.  Responsive Web Design , 2011 .

[48]  Giovanni De Gasperis,et al.  Movement analysis based on virtual reality and 3D depth sensing camera for whole body rehabilitation , 2012, CompIMAGE.

[49]  P. Lang,et al.  International Affective Picture System (IAPS): Instruction Manual and Affective Ratings (Tech. Rep. No. A-4) , 1999 .

[50]  Soo-Young Lee,et al.  Brain–computer interface using fMRI: spatial navigation by thoughts , 2004, Neuroreport.

[51]  Maurice van Keulen,et al.  Hadoop for EEG Storage and Processing: A Feasibility Study , 2014, Brain Informatics and Health.

[52]  Giuseppe Placidi,et al.  A Brain Computer Interface by EEG Signals from Self-induced Emotions , 2017 .

[53]  N. Draper,et al.  Applied Regression Analysis: Draper/Applied Regression Analysis , 1998 .

[54]  Danilo Avola,et al.  Basis for the implementation of an EEG-based single-trial binary brain computer interface through the disgust produced by remembering unpleasant odors , 2015, Neurocomputing.

[55]  N. Birbaumer,et al.  Lower Limb Movement Preparation in Chronic Stroke , 2014, Neurorehabilitation and neural repair.

[56]  Marco Ferrari,et al.  A semi-immersive virtual reality incremental swing balance task activates prefrontal cortex: A functional near-infrared spectroscopy study , 2014, NeuroImage.

[57]  Luigi Cinque,et al.  Overall design and implementation of the virtual glove , 2013, Comput. Biol. Medicine.

[58]  Marco Ferrari,et al.  Prefrontal Cortex Activated Bilaterally by a Tilt Board Balance Task: A Functional Near-Infrared Spectroscopy Study in a Semi-Immersive Virtual Reality Environment , 2013, Brain Topography.

[59]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[60]  Valentina Quaresima,et al.  A low-cost real time virtual system for postural stability assessment at home , 2014, Comput. Methods Programs Biomed..

[61]  Maja Stikic,et al.  EEG-based classification of positive and negative affective states , 2014 .

[62]  Giuseppe Placidi,et al.  Characterization of a SimMechanics Model for a Virtual Glove Rehabilitation System , 2010, CompIMAGE.