Driver drowsiness detection using hybrid convolutional neural network and long short-term memory

Drowsiness and fatigue of the drivers are amongst the significant causes of the car accidents. Every year the number of deaths and fatalities are tremendously increasing due to multifaceted issues and henceforth requires an intelligent processing system for accident avoidance. In relevant with this, an effective driver drowsiness detection system is proposed. The main challenges are robustness of the algorithm towards variation of the human face and real-time processing capability. The first challenge pertaining to the facial variation has been handled well using conventional image processing and hand-craft features of computer vision algorithms. Yet, variations such as facial expression, lighting condition, intra-class variation, and pose variation are additional issues that conventional method failed to address. Deep learning is an alternative solution which provides a better performance by learning features automatically. Thus, this paper proposed a new concept for handling the real-time driver drowsiness detection using the hybrid of convolutional neural network (CNN) and long short-term memory (LSTM). The performance of the system has been tested using the public drowsy driver dataset from ACCV 2016 competition. The results show that it can outperform the former schemes in the literature.

[1]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[2]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[4]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[7]  Ghassan Al-Regib,et al.  TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition , 2017, Signal Process. Image Commun..

[8]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[11]  Gamini Dissanayake,et al.  Driver Drowsiness Classification Using Fuzzy Wavelet-Packet-Based Feature-Extraction Algorithm , 2011, IEEE Transactions on Biomedical Engineering.

[12]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[15]  Fei Pan,et al.  Driver Drowsiness Detection System Based on Feature Representation Learning Using Various Deep Networks , 2016, ACCV Workshops.

[16]  Chiou-Ting Hsu,et al.  MSTN: Multistage Spatial-Temporal Network for Driver Drowsiness Detection , 2016, ACCV Workshops.

[17]  Shang-Hong Lai,et al.  Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network , 2016, ACCV Workshops.

[18]  Brent Lance,et al.  Driver Drowsiness Estimation From EEG Signals Using Online Weighted Adaptation Regularization for Regression (OwARR) , 2017, IEEE Transactions on Fuzzy Systems.

[19]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Kenneth Sundaraj,et al.  Detecting Driver Drowsiness Based on Sensors: A Review , 2012, Sensors.

[21]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[22]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[23]  Bo Cheng,et al.  Driver drowsiness recognition based on computer vision technology , 2012 .

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Manuk Akopyan,et al.  Large-Scale YouTube-8M Video Understanding with Deep Neural Networks , 2017, ArXiv.

[27]  Yongqiang Wang,et al.  Simplifying long short-term memory acoustic models for fast training and decoding , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Sangwook Lee,et al.  Representation Learning, Scene Understanding, and Feature Fusion for Drowsiness Detection , 2016, ACCV Workshops.

[30]  Jordi Torres,et al.  Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks , 2017, ICLR.