Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN

With a focus on fatigue driving detection research, a fully automated driver fatigue status detection algorithm using driving images is proposed. In the proposed algorithm, the multitask cascaded convolutional network (MTCNN) architecture is employed in face detection and feature point location, and the region of interest (ROI) is extracted using feature points. A convolutional neural network, named EM-CNN, is proposed to detect the states of the eyes and mouth from the ROI images. The percentage of eyelid closure over the pupil over time (PERCLOS) and mouth opening degree (POM) are two parameters used for fatigue detection. Experimental results demonstrate that the proposed EM-CNN can efficiently detect driver fatigue status using driving images. The proposed algorithm EM-CNN outperforms other CNN-based methods, i.e., AlexNet, VGG-16, GoogLeNet, and ResNet50, showing accuracy and sensitivity rates of 93.623% and 93.643%, respectively.

[1]  Xinmei Tian,et al.  Multi-organ plant identification with multi-column deep convolutional neural networks , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[2]  Shuo Yang,et al.  WIDER FACE: A Face Detection Benchmark , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jun Li,et al.  Face Fatigue Detection Method Based on MTCNN and Machine Vision , 2019, Advances in Intelligent Systems and Computing.

[4]  Harald Sack,et al.  Fine tuning CNNS with scarce training data — Adapting imagenet to art epoch classification , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[5]  Mahesh M. Bundele,et al.  Design & analysis of k-means algorithm for cognitive fatigue detection in vehicular driver using oximetry pulse signal , 2015, 2015 International Conference on Computer, Communication and Control (IC4).

[6]  Xiaofeng Li,et al.  Cascade and Fusion of Multitask Convolutional Neural Networks for Detection of Thyroid Nodules in Contrast-Enhanced CT , 2019, Comput. Intell. Neurosci..

[7]  Rong Jian,et al.  Discriminating Threshold of Driving Fatigue Based on the Electroencephalography Sample Entropy by Receiver Operating Characteristic Curve Analysis , 2013 .

[8]  M. K. Venkatesha,et al.  A Two Fold Expert System for Yawning Detection , 2016 .

[9]  Naif Alajlan,et al.  Deep Learning Approach for Car Detection in UAV Imagery , 2017, Remote. Sens..

[10]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[11]  Ridha Soua,et al.  Recent Trends in Driver Safety Monitoring Systems: State of the Art and Challenges , 2017, IEEE Transactions on Vehicular Technology.

[12]  Ling Huang,et al.  Monitoring drivers’ sleepy status at night based on machine vision , 2016, Multimedia Tools and Applications.

[13]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[14]  Bao-Liang Lu,et al.  Driving fatigue detection with fusion of EEG and forehead EOG , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[15]  Xu Tang,et al.  PyramidBox: A Context-assisted Single Shot Face Detector , 2018, ECCV.

[16]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[17]  Jun-Juh Yan,et al.  Real-Time Driver Drowsiness Detection System Based on PERCLOS and Grayscale Image Processing , 2016, 2016 International Symposium on Computer, Consumer and Control (IS3C).

[18]  Aamir Saeed Malik,et al.  A Review on EEG-Based Automatic Sleepiness Detection Systems for Driver , 2018, IEEE Access.

[19]  Hui Zhang,et al.  A study on driver fatigue recognition based on SVM method , 2017, 2017 4th International Conference on Transportation Information and Safety (ICTIS).

[20]  Vinod Kulathumani,et al.  Real-time drowsiness detection using wearable, lightweight brain sensing headbands , 2017 .

[21]  Dan Liu,et al.  Driving Fatigue Detection Based on EEG Signal , 2015, 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC).

[22]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Lee Boon-Leng,et al.  Mobile-based wearable-type of driver fatigue detection by GSR and EMG , 2015, TENCON 2015 - 2015 IEEE Region 10 Conference.

[24]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[25]  Gebraeil Nasl Saraji,et al.  Presenting a model for dynamic facial expression changes in detecting drivers’ drowsiness , 2015, Electronic physician.

[26]  Marc Carreras,et al.  Two-step gradient-based reinforcement learning for underwater robotics behavior learning , 2013, Robotics Auton. Syst..

[27]  Pengfei Duan,et al.  Multi-column Deep Neural Network for Offline Arabic Handwriting Recognition , 2017, ICANN.

[28]  Miguel Ángel Sotelo,et al.  The Experience of DRIVERTIVE-DRIVERless cooperaTIve VEhicle-Team in the 2016 GCDC , 2018, IEEE Transactions on Intelligent Transportation Systems.

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Bernt Schiele,et al.  A Convnet for Non-maximum Suppression , 2015, GCPR.

[33]  Maneesha V. Ramesh,et al.  Intelligent Steering Wheel Sensor Network for Real-Time Monitoring and Detection of Driver Drowsiness , 2011 .