Brain dynamic states analysis based on 3D convolutional neural network

Drowsiness driving is one major factor of traffic accident. Monitoring the changes of brain signals provides an effective and direct way for drowsiness detection. One 3D convolutional neural network (3D CNN)-based forecasting system has been proposed to monitor electroencephalography (EEG) signals and predict fatigue level during driving. The limited weight sharing and channel-wise convolution were both applied to extract the significant phenomenon in various frequency bands of brain signals and the spatial information of EEG channel location, respectively. The proposed 3D CNN with limited weight sharing and channel-wise convolution has been demonstrated to predict reaction time (RT) of driving with low root mean square error (RMSE) through the brain dynamics. This proposed approach outperforms with the state-of-the-art algorithms, such as traditional CNN, Neural Network (NN), and support vector regression (SVR). Compared with traditional CNN and Artificial Neural Network, the RMSE of 3D CNN-based RT prediction has been improved 9.5% (RMSE from 0.6322 to 0.5720) and 8% (RMSE from 0.6217 to 0.5720), respectively. We envision that this study might open a new branch between deep learning application in neuro-cognitive analysis and real world application.

[1]  Roman Rosipal,et al.  EEG-Based Drivers' Drowsiness Monitoring Using a Hierarchical Gaussian Mixture Model , 2007, HCI.

[2]  ImageNet Classification with Deep Convolutional Neural , 2013 .

[3]  Pasin Israsena,et al.  EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation , 2014, TheScientificWorldJournal.

[4]  Jacques Bergeron,et al.  Monotony of road environment and driver fatigue: a simulator study. , 2003, Accident; analysis and prevention.

[5]  Chin-Teng Lin,et al.  Assessment of Mental Fatigue: An EEG-Based Forecasting System for Driving Safety , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[6]  William J. Horrey,et al.  The challenges and opportunities of technological approaches to fatigue management. , 2011, Accident; analysis and prevention.

[7]  Amit Shahar,et al.  Risk perception of driving as a function of advanced training aimed at recognizing and handling risks in demanding driving situations. , 2008, Accident; analysis and prevention.

[8]  Tzyy-Ping Jung,et al.  EEG-Based Attention Tracking During Distracted Driving , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Chun-Hsiang Chuang,et al.  An EEG-Based Fatigue Detection and Mitigation System , 2016, Int. J. Neural Syst..

[10]  William J Horrey,et al.  Future directions in fatigue and safety research. , 2011, Accident; analysis and prevention.

[11]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Chris Berka,et al.  Drowsiness/alertness algorithm development and validation using synchronized EEG and cognitive performance to individualize a generalized model , 2011, Biological Psychology.

[13]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Yan Wu,et al.  Convolutional deep belief networks for feature extraction of EEG signal , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  R. Heinzer,et al.  [Driver fatigue]. , 2007, Revue medicale suisse.

[17]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[18]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[19]  Yufei Huang,et al.  Prediction of driver's drowsy and alert states from EEG signals with deep learning , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).