EEG-based Cross-Subject Driver Drowsiness Recognition with an Interpretable Convolutional Neural Network

In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this paper, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%72.68% and state-of-the-art deep learning methods of 71.75%75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., Alpha spindles, as strong indicators of drowsiness across different subjects. In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.

[1]  J. Baumeister,et al.  Influence of phosphatidylserine on cognitive performance and cortical activity after induced stress , 2008, Nutritional neuroscience.

[2]  Ivan Volosyak,et al.  Multiple Channel Detection of Steady-State Visual Evoked Potentials for Brain-Computer Interfaces , 2007, IEEE Transactions on Biomedical Engineering.

[3]  Evangelos Bekiaris,et al.  Using EEG spectral components to assess algorithms for detecting fatigue , 2009, Expert Syst. Appl..

[4]  S. Makeig,et al.  Imaging human EEG dynamics using independent component analysis , 2006, Neuroscience & Biobehavioral Reviews.

[5]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Tzyy-Ping Jung,et al.  EEG-based drowsiness estimation for safety driving using independent component analysis , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[7]  Jianfeng Hu,et al.  Automated detection of driver fatigue based on EEG signals using gradient boosting decision tree model , 2018, Cognitive Neurodynamics.

[8]  Xinmin Wang,et al.  EEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Chin-Teng Lin,et al.  Multi-channel EEG recordings during a sustained-attention driving task , 2018, Scientific Data.

[10]  Jianhui Zhao,et al.  Cascaded Convolutional Neural Network with Attention Mechanism for Mobile EEG-based Driver Drowsiness Detection System , 2019, 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[11]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Michael Schrauf,et al.  EEG alpha spindle measures as indicators of driver fatigue under real traffic conditions , 2011, Clinical Neurophysiology.

[13]  J. Wolpaw,et al.  EMG contamination of EEG: spectral and topographical characteristics , 2003, Clinical Neurophysiology.

[14]  Wei Wu,et al.  Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs , 2007, IEEE Transactions on Biomedical Engineering.

[15]  Cuntai Guan,et al.  Inter-subject transfer learning with an end-to-end deep convolutional neural network for EEG-based BCI , 2019, Journal of neural engineering.

[16]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  Olga Sourina,et al.  EEG-Based Cross-Subject Mental Fatigue Recognition , 2019, 2019 International Conference on Cyberworlds (CW).

[19]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[20]  G. Kecklund,et al.  Manifest sleepiness and the spectral content of the EEG during shift work. , 1991, Sleep.

[21]  G. Torres,et al.  Electroencephalogram Mapping of Brain States , 2014 .

[22]  Selim R. Benbadis,et al.  Normal Sleep EEG , 2010 .

[23]  Olga Sourina,et al.  Subject-Independent Drowsiness Recognition from Single-Channel EEG with an Interpretable CNN-LSTM model , 2021, 2021 International Conference on Cyberworlds (CW).

[24]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Junjie Chen,et al.  The detection of epileptic seizure signals based on fuzzy entropy , 2015, Journal of Neuroscience Methods.

[26]  Olga Sourina,et al.  A Compact and Interpretable Convolutional Neural Network for Cross-Subject Driver Drowsiness Detection from Single-Channel EEG , 2021, Methods.

[27]  Olga Sourina,et al.  Inter-subject transfer learning for EEG-based mental fatigue recognition , 2020, Adv. Eng. Informatics.

[28]  Yoshua Bengio,et al.  Speaker Recognition from Raw Waveform with SincNet , 2018, 2018 IEEE Spoken Language Technology Workshop (SLT).

[29]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[30]  Guojun Dai,et al.  EEG classification of driver mental states by deep learning , 2018, Cognitive Neurodynamics.

[31]  Tzyy-Ping Jung,et al.  EEG-Based Subject- and Session-independent Drowsiness Detection: An Unsupervised Approach , 2008, EURASIP J. Adv. Signal Process..

[32]  Lauren C. Frey,et al.  Electroencephalography (EEG): An Introductory Text and Atlas of Normal and Abnormal Findings in Adults, Children, and Infants , 2016 .

[33]  W. Klimesch.  EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[34]  Stephen M. Gordon,et al.  EEGNet: A Compact Convolutional Neural Network for EEG-based Brain-Computer Interfaces , 2021 .

[35]  Nauman Aslam,et al.  Filtering techniques for channel selection in motor imagery EEG applications: a survey , 2019, Artificial Intelligence Review.

[36]  A. Craig,et al.  Driver fatigue: electroencephalography and psychological assessment. , 2002, Psychophysiology.

[37]  Yibo Li,et al.  Analysis of Feature Fatigue EEG Signals Based on Wavelet Entropy , 2018, Int. J. Pattern Recognit. Artif. Intell..

[38]  Tzyy-Ping Jung,et al.  Toward Drowsiness Detection Using Non-hair-Bearing EEG-Based Brain-Computer Interfaces , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  J. Crowcroft,et al.  Automatic epileptic seizure detection in EEGs based on optimized sample entropy and extreme learning machine , 2012, Journal of Neuroscience Methods.

[40]  Silvia Fantozzi,et al.  Interpretable and lightweight convolutional neural network for EEG decoding: Application to movement execution and imagination , 2020, Neural Networks.

[41]  M A Guevara,et al.  Changes in the waking EEG as a consequence of sleep and sleep deprivation. , 1992, Sleep.

[42]  Yijun Wang,et al.  Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[43]  Anilkumar V. Nandi,et al.  Deep Convolution Neural Network-Based Feature Learning Model for EEG Based Driver Alert/Drowsy State Detection , 2019, SoCPaR.

[44]  Evgin Goceri,et al.  CapsNet topology to classify tumours from brain images and comparative evaluation , 2020, IET Image Process..

[45]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[46]  Kara van de Graaf.  Manifest , 2014, Migratory Sound.

[47]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[48]  Konda Reddy Mopuri,et al.  CNN Fixations: An Unraveling Approach to Visualize the Discriminative Image Regions , 2019, IEEE Transactions on Image Processing.

[49]  J. Shaw,et al.  The brain's alpha rhythms and the mind : a review of classical and modern studies of the alpha rhythm component of the electroencephalogram with commentaries on associated neuroscience and neuropsychology , 2003 .