A Novel Real-time Driver Monitoring System Based on Deep Convolutional Neural Network

In this paper, we propose a novel real-time driver monitoring system based on deep convolutional neural network. Our system can efficiently detect head and facial features. It also allows to accurately estimate the distance between driver's head and camera, and vertical and horizontal rotation angles of head. Our work is inspired by the third version of YOLO (YOLOv3), a well-known objects detection algorithm. We have successfully introduced important improvements on YOLOv3 to further fasten the detection speed for excellent accuracy. Indeed, we reduced the depth and width of the backbone network of YOLO. We called this refined network HeadNet. Then, we implemented K-means algorithm to find appropriate anchors for head and facial features. Furthermore, we designed a new four-layer network knows as OrganNet to detect facial features. Our experiments show that our system has a high detection accuracy of 96.2% to detect head, and 91.5% to detect facial features, with a fast detection speed of 18 frames per second.

[1]  Neil K Chaudhary,et al.  Effect of Passenger Presence on Older Drivers’ Risk of Fatal Crash Involvement , 2014, Traffic injury prevention.

[2]  Azzedine Boukerche,et al.  A real-time lane marking localization, tracking and communication system , 2016, Comput. Commun..

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[5]  Azzedine Boukerche,et al.  Lane detection and tracking system based on the MSER algorithm, hough transform and kalman filter , 2014, MSWiM '14.

[6]  Bin Li,et al.  Real Time Eye Detector with Cascaded Convolutional Neural Networks , 2018, Appl. Comput. Intell. Soft Comput..

[7]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Peter Corcoran,et al.  Convolutional Neural Network Implementation for Eye-Gaze Estimation on Low-Quality Consumer Imaging Systems , 2019, IEEE Transactions on Consumer Electronics.

[9]  Azzedine Boukerche,et al.  Real-Time Vehicle Make and Model Recognition Based on a Bag of SURF Features , 2016, IEEE Transactions on Intelligent Transportation Systems.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Pablo Lardelli-Claret,et al.  The influence of passengers on the risk of the driver causing a car collision in Spain. Analysis of collisions from 1990 to 1999. , 2004, Accident; analysis and prevention.

[12]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[13]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Thierry Derrmann,et al.  Driver Behavior Profiling Using Smartphones: A Low-Cost Platform for Driver Monitoring , 2015, IEEE Intelligent Transportation Systems Magazine.

[15]  Li-Jia Li,et al.  Multi-view Face Detection Using Deep Convolutional Neural Networks , 2015, ICMR.

[16]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Azzedine Boukerche,et al.  Extending the Detection Range of Vision-Based Vehicular Instrumentation , 2016, IEEE Transactions on Instrumentation and Measurement.

[19]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[20]  Clifford Nass,et al.  Assessing the Effectiveness of Interactive Media in Improving Drowsy Driver Safety , 2008, Hum. Factors.

[21]  Azzedine Boukerche,et al.  Animal-Vehicle Collision Mitigation System for Automated Vehicles , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Shan Lin,et al.  WiDet: Wi-Fi Based Device-Free Passive Person Detection with Deep Convolutional Neural Networks , 2018, MSWiM.

[23]  Masakazu Matsugu,et al.  Subject independent facial expression recognition with robust face detection using a convolutional neural network , 2003, Neural Networks.

[24]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[26]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[28]  Azzedine Boukerche,et al.  Design of lane keeping assist system for autonomous vehicles , 2015, 2015 7th International Conference on New Technologies, Mobility and Security (NTMS).

[29]  Shoji Kawahito,et al.  A real-time driver monitoring system using a high sensitivity camera , 2019, BiOS.

[30]  Azzedine Boukerche,et al.  Design of traffic sign detection, recognition, and transmission systems for smart vehicles , 2013, IEEE Wireless Communications.