Vigilance Estimation Using a Wearable EOG Device in Real Driving Environment

Vigilance decrement in driving tasks has been reported to be a major factor in fatal accidents and could severely endanger public transportation safety. However, efficient approaches for estimating vigilance in real driving environment are still lacking. In this paper, we propose a novel approach for implementing continuous vigilance estimation using forehead electrooculograms (EOGs) acquired by wearable dry electrodes in both simulated and real driving environments. To improve the feasibility of this approach for real-world applications, a forehead EOG-based electrode placement with only four electrodes is designed. Flexible dry electrodes and an acquisition board are integrated as a wearable device for recording EOGs. Twenty and ten subjects participated in the simulated and real-world driving environment experiments, respectively. Accurate eye movement parameters from eye-tracking glasses are extracted to calculate the PERCLOS index for vigilance annotation. This is because the vigilance state is a temporally dynamic process, and a continuous conditional random field and a continuous conditional neural field are introduced to construct more accurate vigilance estimation models. To evaluate the efficiency of our system, systematic experiments are performed in real scenarios under various illumination and weather conditions following laboratory simulations as preliminary studies. The experimental results demonstrate that the wearable dry electrode prototype, which has a relatively comfortable forehead setup, can efficiently capture vigilance dynamics. The best mean correlation coefficients achieved by our proposed approach are 71.18% and 66.20% in laboratory simulations and real-world driving environments, respectively. The cross-environment experiments are performed to evaluate the simulated-to-real generalization and a best mean correlation coefficient of 53.96% is achieved.

[1]  Sangtae Ahn,et al.  Exploring Neuro-Physiological Correlates of Drivers' Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data , 2016, Front. Hum. Neurosci..

[2]  Bao-Liang Lu,et al.  EEG-based vigilance estimation using extreme learning machines , 2013, Neurocomputing.

[3]  Tianwei Shi,et al.  Real-Time EEG-Based Detection of Fatigue Driving Danger for Accident Prediction , 2015, Int. J. Neural Syst..

[4]  M. Posner Measuring Alertness , 2008, Annals of the New York Academy of Sciences.

[5]  B. Oken,et al.  Vigilance, alertness, or sustained attention: physiological basis and measurement , 2006, Clinical Neurophysiology.

[6]  Guangli Li,et al.  Novel passive ceramic based semi-dry electrodes for recording electroencephalography signals from the hairy scalp , 2016 .

[7]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[8]  Peter Robinson,et al.  Dimensional affect recognition using Continuous Conditional Random Fields , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[9]  Longchun Wang,et al.  PDMS-Based Low Cost Flexible Dry Electrode for Long-Term EEG Measurement , 2012, IEEE Sensors Journal.

[10]  Bao-Liang Lu,et al.  A novel method for EOG features extraction from the forehead , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[11]  Juan Zhou,et al.  Spontaneous eyelid closures link vigilance fluctuation with fMRI dynamic connectivity states , 2016, Proceedings of the National Academy of Sciences.

[12]  Jing Quan Liu,et al.  Parylene-based flexible dry electrode for bioptential recording , 2016 .

[13]  Jian Peng,et al.  Conditional Neural Fields , 2009, NIPS.

[14]  Tzyy-Ping Jung,et al.  Dry-Contact and Noncontact Biopotential Electrodes: Methodological Review , 2010, IEEE Reviews in Biomedical Engineering.

[15]  Roozbeh Jafari,et al.  Design Principles and Dynamic Front End Reconfiguration for Low Noise EEG Acquisition With Finger Based Dry Electrodes , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[16]  Ronen Talmon,et al.  Empirical intrinsic geometry for nonlinear modeling and time series filtering , 2013, Proceedings of the National Academy of Sciences.

[17]  E R Braver,et al.  The sleep of long-haul truck drivers. , 1998, The New England journal of medicine.

[18]  Deepak Ganesan,et al.  Domain adaptation methods for improving lab-to-field generalization of cocaine detection using wearable ECG , 2016, UbiComp.

[19]  Bao-Liang Lu,et al.  A novel approach to driving fatigue detection using forehead EOG , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[20]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[21]  Tzyy-Ping Jung,et al.  Real-time neuroimaging and cognitive monitoring using wearable dry EEG , 2015, IEEE Transactions on Biomedical Engineering.

[22]  Chin-Teng Lin,et al.  Identifying changes in EEG information transfer during drowsy driving by transfer entropy , 2015, Front. Hum. Neurosci..

[23]  Richard D. Jones,et al.  EEG-Based Lapse Detection With High Temporal Resolution , 2007, IEEE Transactions on Biomedical Engineering.

[24]  Zhiwei Zhu,et al.  Real-time nonintrusive monitoring and prediction of driver fatigue , 2004, IEEE Transactions on Vehicular Technology.

[25]  Bor-Shyh Lin,et al.  Novel Active Comb-Shaped Dry Electrode for EEG Measurement in Hairy Site , 2015, IEEE Transactions on Biomedical Engineering.

[26]  Peter Robinson,et al.  Continuous Conditional Neural Fields for Structured Regression , 2014, ECCV.

[27]  Karan Singh,et al.  Learning Linear Dynamical Systems via Spectral Filtering , 2017, NIPS.

[28]  Sang-Hoon Lee,et al.  Self-Adhesive and Capacitive Carbon Nanotube-Based Electrode to Record Electroencephalograph Signals From the Hairy Scalp , 2016, IEEE Transactions on Biomedical Engineering.

[29]  Tao Zhang,et al.  Drowsiness Detection by Bayesian-Copula Discriminant Classifier Based on EEG Signals During Daytime Short Nap , 2017, IEEE Transactions on Biomedical Engineering.

[30]  Gernot R. Müller-Putz,et al.  Evaluation of Different EEG Acquisition Systems Concerning Their Suitability for Building a Brain–Computer Interface: Case Studies , 2016, Front. Neurosci..

[31]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[32]  R. Shumway,et al.  Time Series Regression and Exploratory Data Analysis , 2011 .

[33]  Gamini Dissanayake,et al.  Driver Drowsiness Classification Using Fuzzy Wavelet-Packet-Based Feature-Extraction Algorithm , 2011, IEEE Transactions on Biomedical Engineering.

[34]  Gregor Leicht,et al.  EEG-vigilance and BOLD effect during simultaneous EEG/fMRI measurement , 2009, NeuroImage.

[35]  Nilanjan Sarkar,et al.  Cognitive Load Measurement in a Virtual Reality-Based Driving System for Autism Intervention , 2017, IEEE Transactions on Affective Computing.

[36]  Eric Larson,et al.  Leveraging anatomical information to improve transfer learning in brain–computer interfaces , 2015, Journal of neural engineering.

[37]  Jae Gwan Kim,et al.  Utilization of a combined EEG/NIRS system to predict driver drowsiness , 2017, Scientific Reports.

[38]  T. Åkerstedt,et al.  Asleep at the Wheel-The Road to Addressing Drowsy Driving. , 2017, Sleep.

[39]  Chun-Hsiang Chuang,et al.  Wireless and Wearable EEG System for Evaluating Driver Vigilance , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[40]  M. Ferrara,et al.  How much sleep do we need? , 2001, Sleep medicine reviews.

[41]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[42]  Benjamin Blankertz,et al.  EEG predictors of covert vigilant attention , 2014, Journal of neural engineering.

[43]  Bao-Liang Lu,et al.  Evaluating driving fatigue detection algorithms using eye tracking glasses , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[44]  Bao-Liang Lu,et al.  Driving fatigue detection with fusion of EEG and forehead EOG , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[45]  M. J. Green The sleep of long-haul truck drivers. , 1998, The New England journal of medicine.

[46]  Sang-Hoon Lee,et al.  CNT/PDMS Composite Flexible Dry Electrodesfor Long-Term ECG Monitoring , 2012, IEEE Transactions on Biomedical Engineering.

[47]  Bao-Liang Lu,et al.  An EOG-based Vigilance Estimation Method Applied for Driver Fatigue Detection , 2015 .

[48]  Ivan Ho Mien,et al.  Heart rate variability can be used to estimate sleepiness-related decrements in psychomotor vigilance during total sleep deprivation. , 2012, Sleep.

[49]  F. Gino,et al.  Cognitive fatigue influences students’ performance on standardized tests , 2016, Proceedings of the National Academy of Sciences.

[50]  B. Oken,et al.  Sleeping and driving: Not a safe dual-task , 2007, Clinical Neurophysiology.

[51]  Udo Trutschel,et al.  PERCLOS: An Alertness Measure of the Past , 2017 .

[52]  Chin-Teng Lin,et al.  Development of Wireless Brain Computer Interface With Embedded Multitask Scheduling and its Application on Real-Time Driver's Drowsiness Detection and Warning , 2008, IEEE Transactions on Biomedical Engineering.

[53]  T. Jung,et al.  Can arousing feedback rectify lapses in driving? Prediction from EEG power spectra , 2013, Journal of neural engineering.

[54]  Boris Murmann,et al.  Highly stretchable polymer semiconductor films through the nanoconfinement effect , 2017, Science.

[55]  Keiichi Uchimura,et al.  Driver Inattention Monitoring System for Intelligent Vehicles: A Review , 2009, IEEE Transactions on Intelligent Transportation Systems.

[56]  T. Jung,et al.  Dry and Noncontact EEG Sensors for Mobile Brain–Computer Interfaces , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[57]  Bao-Liang Lu,et al.  Transfer Components Between Subjects for EEG-based Driving Fatigue Detection , 2015, ICONIP.

[58]  Motoaki Kawanabe,et al.  Learning a common dictionary for subject-transfer decoding with resting calibration , 2015, NeuroImage.

[59]  P. R. Davidson,et al.  Detection of lapses in responsiveness from the EEG , 2011, Journal of neural engineering.

[60]  Bao-Liang Lu,et al.  A multimodal approach to estimating vigilance using EEG and forehead EOG , 2016, Journal of neural engineering.

[61]  Gert Cauwenberghs,et al.  Integrated Circuits and Electrode Interfaces for Noninvasive Physiological Monitoring , 2014, IEEE Transactions on Biomedical Engineering.

[62]  Xiaoping Chen,et al.  EOG-based drowsiness detection using convolutional neural networks , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[63]  Joong Hoon Lee,et al.  CNT/PDMS-based canal-typed ear electrodes for inconspicuous EEG recording , 2014, Journal of neural engineering.

[64]  Chunsheng Yang,et al.  A novel passive electrode based on porous Ti for EEG recording , 2016 .

[65]  P. Rossini,et al.  Antero-posterior functional coupling at sleep onset: changes as a function of increased sleep pressure , 2005, Brain Research Bulletin.

[66]  S. Johnstone,et al.  Test-retest reliability of a single-channel, wireless EEG system. , 2016, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[67]  Chrysoula Kourtidou-Papadeli,et al.  Monitoring sleepiness with on-board electrophysiological recordings for preventing sleep-deprived traffic accidents , 2007, Clinical Neurophysiology.

[68]  T O Zander,et al.  Context-aware brain–computer interfaces: exploring the information space of user, technical system and environment , 2012, Journal of neural engineering.

[69]  Jonathan Smallwood,et al.  Subjective experience and the attentional lapse: Task engagement and disengagement during sustained attention , 2004, Consciousness and Cognition.

[70]  Yijun Wang,et al.  A Self-Wetting Paper Electrode for Ubiquitous Bio-Potential Monitoring , 2017, IEEE Sensors Journal.

[71]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[72]  Tzyy-Ping Jung,et al.  EEG-based drowsiness estimation for safety driving using independent component analysis , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[73]  Miguel Ángel Sotelo,et al.  Real-time system for monitoring driver vigilance , 2004, Proceedings of the IEEE International Symposium on Industrial Electronics, 2005. ISIE 2005..

[74]  Dimitrios Tzovaras,et al.  Fuzzy Fusion of Eyelid Activity Indicators for Hypovigilance-Related Accident Prediction , 2008, IEEE Transactions on Intelligent Transportation Systems.