Detecting Driver Drowsiness as an Anomaly Using LSTM Autoencoders

In this paper, an LSTM autoencoder-based architecture is utilized for drowsiness detection with ResNet-34 as feature extractor. The problem is considered as anomaly detection for a single subject; therefore, only the normal driving representations are learned and it is expected that drowsiness representations, yielding higher reconstruction losses, are to be distinguished according to the knowledge of the network. In our study, the confidence levels of normal and anomaly clips are investigated through the methodology of label assignment such that training performance of LSTM autoencoder and interpretation of anomalies encountered during testing are analyzed under varying confidence rates. Our method is experimented on NTHU-DDD and benchmarked with a state-of-the-art anomaly detection method for driver drowsiness. Results show that the proposed model achieves detection rate of 0.8740 area under curve (AUC) and is able to provide significant improvements on certain scenarios.

[1]  Bing-Ting Dong,et al.  An On-board Monitoring System for Driving Fatigue and Distraction Detection , 2021, 2021 22nd IEEE International Conference on Industrial Technology (ICIT).

[2]  Bin Zhang,et al.  Robust Two-Stream Multi-Feature Network for Driver Drowsiness Detection , 2020, ArXiv.

[3]  Hang Xu,et al.  Driver Anomaly Detection: A Dataset and Contrastive Learning Approach , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[4]  Bumsub Ham,et al.  Learning Memory-Guided Normality for Anomaly Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[6]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Shenghua Gao,et al.  Remembering history with convolutional LSTM for anomaly detection , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[8]  Ye-Hoon Kim,et al.  Real-Time Driver Drowsiness Detection for Embedded System Using Model Compression of Deep Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Shang-Hong Lai,et al.  Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network , 2016, ACCV Workshops.

[10]  Chiou-Ting Hsu,et al.  MSTN: Multistage Spatial-Temporal Network for Driver Drowsiness Detection , 2016, ACCV Workshops.

[11]  Sangwook Lee,et al.  Representation Learning, Scene Understanding, and Feature Fusion for Drowsiness Detection , 2016, ACCV Workshops.

[12]  Xuan-Phung Huynh,et al.  Detection of Driver Drowsiness Using 3D Deep Neural Network and Semi-Supervised Gradient Boosting Machine , 2016, ACCV Workshops.

[13]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[14]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Luis Miguel Bergasa,et al.  Vision-based drowsiness detector for real driving conditions , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[19]  Simone Benedetto,et al.  Driver workload and eye blink duration , 2011 .

[20]  Dick de Waard,et al.  Monitoring drivers' mental workload in driving simulators using physiological measures. , 2010, Accident; analysis and prevention.

[21]  Gwen Littlewort,et al.  Drowsy Driver Detection Through Facial Movement Analysis , 2007, ICCV-HCI.

[22]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[23]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[24]  J R Treat,et al.  TRI-LEVEL STUDY OF THE CAUSES OF TRAFFIC ACCIDENTS: FINAL REPORT , 1979 .

[25]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.