Decoding Event-related Potential from Ear-EEG Signals based on Ensemble Convolutional Neural Networks in Ambulatory Environment

Recently, practical brain-computer interface is actively carried out, especially, in an ambulatory environment. However, the electroencephalography (EEG) signals are distorted by movement artifacts and electromyography signals when users are moving, which make hard to recognize human intention. In addition, as hardware issues are also challenging, ear-EEG has been developed for practical brain-computer interface and has been widely used. In this paper, we proposed ensemble-based convolutional neural networks in ambulatory environment and analyzed the visual event-related potential responses in scalp- and ear-EEG in terms of statistical analysis and brain-computer interface performance. The brain-computer interface performance deteriorated as 3-14% when walking fast at 1.6 m/s. The proposed methods showed 0.728 in average of the area under the curve. The proposed method shows robust to the ambulatory environment and imbalanced data as well.

[1]  John Williamson,et al.  A High Performance Spelling System based on EEG-EOG Signals With Visual Feedback , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Jane You,et al.  Hybrid Classifier Ensemble for Imbalanced Data , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Klaus-Robert Müller,et al.  A lower limb exoskeleton control system based on steady state visual evoked potentials , 2015, Journal of neural engineering.

[4]  Daniel P Ferris,et al.  Dual-electrode motion artifact cancellation for mobile electroencephalography , 2018, Journal of neural engineering.

[5]  J. Contreras-Vidal,et al.  Real-time EEG-based brain-computer interface to a virtual avatar enhances cortical involvement in human treadmill walking , 2017, Scientific Reports.

[6]  Heung-Il Suk,et al.  Subject and class specific frequency bands selection for multiclass motor imagery classification , 2011, Int. J. Imaging Syst. Technol..

[7]  S. Debener,et al.  Concealed, Unobtrusive Ear-Centered EEG Acquisition: cEEGrids for Transparent EEG , 2017, Front. Hum. Neurosci..

[8]  Seong-Whan Lee,et al.  Network Properties in Transitions of Consciousness during Propofol-induced Sedation , 2017, Scientific Reports.

[9]  Jose L. Contreras-Vidal,et al.  Unscented Kalman filter for neural decoding of human treadmill walking from non-invasive electroencephalography , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[10]  Klaus-Robert Müller,et al.  An Efficient ERP-Based Brain-Computer Interface Using Random Set Presentation and Face Familiarity , 2014, PloS one.

[11]  Minji Lee,et al.  Decoding Visual Responses based on Deep Neural Networks with Ear-EEG Signals , 2020, 2020 8th International Winter Conference on Brain-Computer Interface (BCI).

[12]  Siamac Fazli,et al.  Subject-dependent classification for robust idle state detection using multi-modal neuroimaging and data-fusion techniques in BCI , 2015, Pattern Recognit..

[13]  Preben Kidmose,et al.  A Study of Evoked Potentials From Ear-EEG , 2013, IEEE Transactions on Biomedical Engineering.

[14]  Seong-Whan Lee,et al.  Decoding Three-Dimensional Trajectory of Executed and Imagined Arm Movements From Electroencephalogram Signals , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Rui Liu,et al.  Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification , 2019, Inf. Sci..

[16]  S. Debener,et al.  Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear , 2015, Scientific Reports.

[17]  John Williamson,et al.  Mental fatigue in central-field and peripheral-field steady-state visually evoked potential and its effects on event-related potential responses , 2018, Neuroreport.

[18]  Seong-Whan Lee,et al.  Subject-Independent Brain–Computer Interfaces Based on Deep Convolutional Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Ji-Hoon Jeong,et al.  OpenBMI: A real-time data analysis toolbox for Brain-Machine Interfaces , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[20]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[21]  Thierry Dutoit,et al.  Optimizing the Performances of a P300-Based Brain–Computer Interface in Ambulatory Conditions , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[22]  Klaus-Robert Müller,et al.  The Berlin Brain-Computer Interface (BBCI) – towards a new communication channel for online control in gaming applications , 2007, Multimedia Tools and Applications.

[23]  Kup-Sze Choi,et al.  A Deep-Ensemble-Level-Based Interpretable Takagi–Sugeno–Kang Fuzzy Classifier for Imbalanced Data , 2020, IEEE Transactions on Cybernetics.

[24]  Seong-Whan Lee,et al.  Error Correction Regression Framework for Enhancing the Decoding Accuracies of Ear-EEG Brain–Computer Interfaces , 2020, IEEE Transactions on Cybernetics.

[25]  Hyung-Soon Park,et al.  Prefrontal, posterior parietal and sensorimotor network activity underlying speed control during walking , 2015, Front. Hum. Neurosci..

[26]  Hyoung Joong Kim,et al.  A High-Security EEG-Based Login System with RSVP Stimuli and Dry Electrodes , 2016, IEEE Transactions on Information Forensics and Security.

[27]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[28]  Klaus-Robert Müller,et al.  Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  John Williamson,et al.  EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy , 2019, GigaScience.

[30]  Klaus-Robert Müller,et al.  A convolutional neural network for steady state visual evoked potential classification under ambulatory environment , 2017, PloS one.

[31]  Daniel P. Ferris,et al.  Visual Evoked Responses During Standing and Walking , 2010, Front. Hum. Neurosci..

[32]  K. Müller,et al.  Effect of higher frequency on the classification of steady-state visual evoked potentials , 2016, Journal of neural engineering.

[33]  Stephen M. Gordon,et al.  EEGNet: A Compact Convolutional Neural Network for EEG-based Brain-Computer Interfaces , 2021 .

[34]  Seong-Whan Lee,et al.  A Real-Time Movement Artifact Removal Method for Ambulatory Brain-Computer Interfaces , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.