Assessment of the Potential of Wrist-Worn Wearable Sensors for Driver Drowsiness Detection

Drowsy driving imposes a high safety risk. Current systems often use driving behavior parameters for driver drowsiness detection. The continuous driving automation reduces the availability of these parameters, therefore reducing the scope of such methods. Especially, techniques that include physiological measurements seem to be a promising alternative. However, in a dynamic environment such as driving, only non- or minimal intrusive methods are accepted, and vibrations from the roadbed could lead to degraded sensor technology. This work contributes to driver drowsiness detection with a machine learning approach applied solely to physiological data collected from a non-intrusive retrofittable system in the form of a wrist-worn wearable sensor. To check accuracy and feasibility, results are compared with reference data from a medical-grade ECG device. A user study with 30 participants in a high-fidelity driving simulator was conducted. Several machine learning algorithms for binary classification were applied in user-dependent and independent tests. Results provide evidence that the non-intrusive setting achieves a similar accuracy as compared to the medical-grade device, and high accuracies (>92%) could be achieved, especially in a user-dependent scenario. The proposed approach offers new possibilities for human–machine interaction in a car and especially for driver state monitoring in the field of automated driving.

[1]  Andy Adler,et al.  Validation of the Empatica E4 wristband , 2016, 2016 IEEE EMBS International Student Conference (ISC).

[2]  M. H. Ebrahimi,et al.  Determination the Levels of Subjective and Observer Rating of Drowsiness and Their Associations with Facial Dynamic Changes , 2017, Iranian journal of public health.

[3]  Steffen Leonhardt,et al.  Unobtrusive Vital Sign Monitoring in Automotive Environments—A Review , 2018, Sensors.

[4]  Kenneth Sundaraj,et al.  Detecting Driver Drowsiness Based on Sensors: A Review , 2012, Sensors.

[5]  Gabriel Lodewijks,et al.  Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue. , 2020, Journal of safety research.

[6]  Fred S. Switzer,et al.  Lane heading difference: An innovative model for drowsy driving detection using retrospective analysis around curves. , 2015, Accident; analysis and prevention.

[7]  Wan-Young Chung,et al.  Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel , 2014 .

[8]  Wan-Young Chung,et al.  Standalone Wearable Driver Drowsiness Detection System in a Smartwatch , 2016, IEEE Sensors Journal.

[9]  Domenico Grimaldi,et al.  Instrumentation and measurement in medical, biomedical, and healthcare systems , 2016, IEEE Instrum. Meas. Mag..

[10]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[11]  Asifullah Khan,et al.  Detecting Driver Drowsiness in Real Time Through Deep Learning Based Object Detection , 2019, IWANN.

[12]  Mobyen Uddin Ahmed,et al.  Automatic driver sleepiness detection using EEG, EOG and contextual information , 2019, Expert Syst. Appl..

[13]  M. Johns,et al.  A new method for measuring daytime sleepiness: the Epworth sleepiness scale. , 1991, Sleep.

[14]  Anthony D. McDonald,et al.  Real-Time Detection of Drowsiness Related Lane Departures Using Steering Wheel Angle , 2012 .

[15]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[16]  Zuojin Li,et al.  Online Detection of Driver Fatigue Using Steering Wheel Angles for Real Driving Conditions , 2017, Sensors.

[17]  Udo Trutschel,et al.  PERCLOS: An Alertness Measure of the Past , 2017 .

[18]  Corbett Ma,et al.  A drowsiness detection system for pilots: Optalert. , 2009 .

[19]  Andreas V Larentzakis,et al.  Can Wearable Devices Accurately Measure Heart Rate Variability? A Systematic Review , 2018, Folia medica.

[20]  Christer Ahlstrom,et al.  Video-based observer rated sleepiness versus self-reported subjective sleepiness in real road driving , 2015, European Transport Research Review.

[21]  Francesco Rundo,et al.  An Innovative Deep Learning Algorithm for Drowsiness Detection from EEG Signal , 2019, Comput..

[22]  B. Tefft Acute Sleep Deprivation and Risk of Motor Vehicle Crash Involvement , 2016 .

[23]  Björn Peters,et al.  Subjective sleepiness and accident risk avoiding the ecological fallacy , 2006, Journal of sleep research.

[24]  David Sandberg The performance of driver sleepiness indicators as a function of interval length , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[25]  Shan Jiang,et al.  Real-time Driver Drowsiness Detection for Android Application Using Deep Neural Networks Techniques , 2018, ANT/SEIT.

[26]  T. Åkerstedt,et al.  Subjective and objective sleepiness in the active individual. , 1990, The International journal of neuroscience.

[27]  Houxiang Zhang,et al.  A Decentralized Sensor Fusion Approach to Human Fatigue Monitoring in Maritime Operations , 2019, 2019 IEEE 15th International Conference on Control and Automation (ICCA).

[28]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[29]  Ali Nahvi,et al.  Driver drowsiness detection based on classification of surface electromyography features in a driving simulator , 2019, Proceedings of the Institution of Mechanical Engineers. Part H, Journal of engineering in medicine.

[30]  Enrique Medina,et al.  Drowsiness Detection Based on the Analysis of Breathing Rate Obtained from Real-time Image Recognition , 2016 .

[31]  Mika P. Tarvainen,et al.  Kubios HRV - Heart rate variability analysis software , 2014, Comput. Methods Programs Biomed..

[32]  Andreas Riener,et al.  A Robust Drowsiness Detection Method based on Vehicle and Driver Vital Data , 2017, MuC Workshopband.

[33]  Andreas Riener,et al.  Drowsiness Detection and Warning in Manual and Automated Driving: Results from Subjective Evaluation , 2018, AutomotiveUI.

[34]  Timothy H. Monk,et al.  A visual analogue scale technique to measure global vigor and affect , 1989, Psychiatry Research.

[35]  Pablo Laguna,et al.  Drowsiness detection using heart rate variability , 2016, Medical & Biological Engineering & Computing.

[36]  Luís Torgo,et al.  A Survey of Predictive Modeling on Imbalanced Domains , 2016, ACM Comput. Surv..

[37]  Rongrong Fu,et al.  Detection of Driving fatigue by using Noncontact EMG and ECG signals Measurement System , 2014, Int. J. Neural Syst..

[38]  Wei Liu,et al.  Vigilance Estimation Using a Wearable EOG Device in Real Driving Environment , 2020, IEEE Transactions on Intelligent Transportation Systems.

[39]  Bo Cheng,et al.  Driver drowsiness recognition based on computer vision technology , 2012 .

[40]  Houxiang Zhang,et al.  Visual Attention Assessment for Expert-in-the-Loop Training in a Maritime Operation Simulator , 2020, IEEE Transactions on Industrial Informatics.

[41]  Bin Yang,et al.  Drowsiness monitoring by steering and lane data based features under real driving conditions , 2010, 2010 18th European Signal Processing Conference.

[42]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[43]  V. Berge-Cherfaoui,et al.  Driver Drowsiness Measurement Technologies: Current Research, Market Solutions, and Challenges , 2019, International Journal of Intelligent Transportation Systems Research.

[44]  Christian Thiele,et al.  Detecting sleep in drivers during highly automated driving: the potential of physiological parameters , 2019, IET Intelligent Transport Systems.

[45]  Ralf Seepold,et al.  ECG sensor for detection of driver's drowsiness , 2019, KES.

[46]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[47]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[48]  Alina Mashko,et al.  SUBJECTIVE METHODS FOR ASSESSMENT OF DRIVER DROWSINESS , 2017 .

[49]  Colin M. Shapiro,et al.  Stanford Sleepiness Scale (SSS) , 2011 .

[50]  Santokh Singh,et al.  Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey , 2015 .

[51]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[52]  M. Mozumdar,et al.  Driver Drowsiness Detection Algorithms Using Electrocardiogram Data Analysis , 2019, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC).

[53]  Anna Persson,et al.  Heart Rate Variability for Driver Sleepiness Classification in Real Road Driving Conditions* , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[54]  Azizul Azizan,et al.  Non-intrusive Driver Drowsiness Detection based on Face and Eye Tracking , 2019, International Journal of Advanced Computer Science and Applications.

[55]  Cheong-Ghil Kim,et al.  Hybrid Driver Fatigue Detection System Based on Data Fusion with Wearable Sensor Devices , 2015 .

[56]  Gang Li,et al.  Detection of Driver Drowsiness Using Wavelet Analysis of Heart Rate Variability and a Support Vector Machine Classifier , 2013, Sensors.

[57]  Jae-Won Lee,et al.  Using Wearable ECG/PPG Sensors for Driver Drowsiness Detection Based on Distinguishable Pattern of Recurrence Plots , 2019, Electronics.

[58]  Ivan Ho Mien,et al.  Heart rate variability can be used to estimate sleepiness-related decrements in psychomotor vigilance during total sleep deprivation. , 2012, Sleep.

[59]  R. Chervin Epworth sleepiness scale? , 2003, Sleep medicine.

[60]  Hans P A Van Dongen,et al.  Efficient driver drowsiness detection at moderate levels of drowsiness. , 2013, Accident; analysis and prevention.

[61]  Hiroshi Ueno,et al.  Development of drowsiness detection system , 1994, Proceedings of VNIS'94 - 1994 Vehicle Navigation and Information Systems Conference.

[62]  Herlina Abdul Rahim,et al.  Detecting Drowsy Driver Using Pulse Sensor , 2015 .

[63]  Tudor Mitran,et al.  Driver drowsiness detection using ANN image processing , 2017 .

[64]  M. Corbett A drowsiness detection system for pilots: Optalert. , 2009, Aviation, space, and environmental medicine.

[65]  R. M. Marston,et al.  ANTI-SLEEP ALARM , 1970 .

[66]  Anna Anund,et al.  Observer Rated Sleepiness and Real Road Driving: An Explorative Study , 2013, PloS one.

[67]  Klaus Bengler,et al.  Highly automated driving: How to get the driver drowsy and how does drowsiness influence various take-over aspects? , 2017 .

[68]  Ioanna Chouvarda,et al.  EEG and HRV markers of sleepiness and loss of control during car driving , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[69]  Shuyan Hu,et al.  Driver drowsiness detection with eyelid related parameters by Support Vector Machine , 2009, Expert Syst. Appl..

[70]  Abdulkadir Sengur,et al.  An Effective Hybrid Model for EEG-Based Drowsiness Detection , 2019, IEEE Sensors Journal.

[71]  Gyogwon Koo,et al.  Wearable Device-Based System to Monitor a Driver’s Stress, Fatigue, and Drowsiness , 2018, IEEE Transactions on Instrumentation and Measurement.

[72]  Wan-Young Chung,et al.  Wearable driver drowsiness detection system based on biomedical and motion sensors , 2015, 2015 IEEE SENSORS.

[73]  Riad I. Hammoud,et al.  Driver State Monitor from DELPHI , 2005, CVPR.

[74]  Ronald R Knipling,et al.  Vehicle-based drowsy driver detection : current status and future prospects , 1994 .

[75]  Maneesha V. Ramesh,et al.  Real-Time Automated Multiplexed Sensor System for Driver Drowsiness Detection , 2011, 2011 7th International Conference on Wireless Communications, Networking and Mobile Computing.

[76]  Varun Bajaj,et al.  Drowsiness Detection Using Adaptive Hermite Decomposition and Extreme Learning Machine for Electroencephalogram Signals , 2018, IEEE Sensors Journal.

[77]  Elizabeth Sherly,et al.  Real time detection system of driver drowsiness based on representation learning using deep neural networks , 2019, J. Intell. Fuzzy Syst..