Driver drowsiness evaluation by means of thermal infrared imaging: preliminary results

Driver’s drowsiness is one of the major causes of traffic accidents worldwide. An early detection of episodes of sleepiness becomes of fundamental importance for safety purposes. Several studies demonstrated that PERCLOS, that is the percentage of eyelid closure over the pupil across time, is one of the most accurate parameters for drowsiness state assessment. However, since PERCLOS is typically computed from the visible video of the subjects, its evaluation is strictly dependent on the lighting conditions and it is not accessible if the driver wears sunglasses. The objective of this study is to overcome these limitations, evaluating drowsy states using a low-cost and high-resolution thermal infrared technology. Ten sleep-deprived subjects were recruited for the experiment, consisting in one-hour driving task on a driving static simulator. During the experiment, facial skin temperature was recorded by means of the thermal camera Device Alab SmartIr640, together with facial visible videos of the subjects. Relevant thermal features were estimated from facial regions of interest (i.e., nose tip, glabella) whereas PERCLOS was performed on visible videos. Features were extracted over a time window of 30 seconds. A data-driven multivariate machine learning approach based on a three-level Support Vector Classification of the drowsy state (AWAKE class: PERCLOS<0.15, FATIGUE class: 0.150.15) was employed. The average classification accuracy was 0.65±0.09 (mean ± standard deviation). Although preliminary, these results indicate the possibility to assess driver's drowsiness based on facial thermal features, overcoming the limitation related to lighting condition and eyes detection, typical of standard methods.

[1]  Linda Ng Boyle,et al.  Driver stress as influenced by driving maneuvers and roadway conditions , 2007 .

[2]  Rekha Saini,et al.  Driver Drowsiness Detection System and Techniques : A Review , 2014 .

[3]  Ajay Mittal,et al.  Head movement-based driver drowsiness detection: A review of state-of-art techniques , 2016, 2016 IEEE International Conference on Engineering and Technology (ICETECH).

[4]  Nathaniel H. Hunt,et al.  The Appropriate Use of Approximate Entropy and Sample Entropy with Short Data Sets , 2012, Annals of Biomedical Engineering.

[5]  Arcangelo Merla,et al.  Driver Stress State Evaluation by Means of Thermal Imaging: A Supervised Machine Learning Approach Based on ECG Signal , 2020, Applied Sciences.

[6]  Xu Huang,et al.  Driving performance in cold, warm, and thermoneutral environments. , 2003, Applied ergonomics.

[7]  M. Zweig,et al.  Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. , 1993, Clinical chemistry.

[8]  Arcangelo Merla,et al.  New Frontiers for Applications of Thermal Infrared Imaging Devices: Computational Psychopshysiology in the Neurosciences , 2017, Sensors.

[9]  Zuojin Li,et al.  Online Detection of Driver Fatigue Using Steering Wheel Angles for Real Driving Conditions , 2017, Sensors.

[10]  Waleed Al-Nuaimy,et al.  EEG-based Driver Fatigue Detection , 2013, 2013 Sixth International Conference on Developments in eSystems Engineering.

[11]  Pratyush Agarwal,et al.  Driver Drowsiness Detection Techniques: Review , 2019 .

[12]  Hu Hongyu,et al.  Driver Drowsiness Detection Based on Time Series Analysis of Steering Wheel Angular Velocity , 2017, 2017 9th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA).

[13]  B. N. Manu,et al.  Facial features monitoring for real time drowsiness detection , 2016, 2016 12th International Conference on Innovations in Information Technology (IIT).

[14]  Arcangelo Merla,et al.  Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review , 2020 .

[15]  Khairul Azami Sidek,et al.  A Review of ECG Data Acquisition for Driver Drowsiness Detection , 2020 .

[16]  V. Jaiganesh,et al.  A Literature Review on Supervised Machine Learning Algorithms and Boosting Process , 2017 .

[17]  Ali Nahvi,et al.  Monitoring the Variation in Driver Respiration Rate from Wakefulness to Drowsiness: A Non-Intrusive Method for Drowsiness Detection Using Thermal Imaging , 2018 .

[18]  Gerald Matthews,et al.  Age and Gender Differences in Stress Responses during Simulated Driving , 1999 .

[19]  Robert S. Allison,et al.  Thermal Imaging as a Way to Classify Cognitive Workload , 2010, 2010 Canadian Conference on Computer and Robot Vision.

[20]  R. Baevsky,et al.  Heart rate variability analysis: physiological foundations and main methods , 2017 .

[21]  Leo Pauly,et al.  Detection of drowsiness based on HOG features and SVM classifiers , 2015, 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN).

[22]  Boguslaw Cyganek,et al.  Driver's fatigue recognition based on yawn detection in thermal images , 2019, Neurocomputing.

[23]  Aki Vehtari,et al.  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC , 2015, Statistics and Computing.

[24]  Christiane,et al.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. , 2004, Journal international de bioethique = International journal of bioethics.

[25]  Noelia Hernández,et al.  Vision-based drowsiness detector for a realistic driving simulator , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[26]  Georges Dupret,et al.  Bootstrap re-sampling for unbalanced data in supervised learning , 2001, Eur. J. Oper. Res..