Recognizing driver inattention by convolutional neural networks

Driver inattention has long been recognized as the main contributing factors in traffic accidents. Development of intelligent driver assistance systems with embedded functionality of driver vigilance monitoring is therefore an urgent and challenging task. This paper presents a novel system which applies convolutional neural network to automatically learn and predict the state of driver's eye, mouth and ear. The initial inspiration is to predict driver fatigue and distraction by analysing these states. In our works, a CNN model was trained with six classes of labeled data. The Approach was verified using self-specified Driving Dataset, which comprised of four activities, including normal driving, responding to a cell phone call, eating and falling asleep. Experiment results demonstrate that our design achieves a promising performance with a overall accuracy of 95.56% in classifying six states of the driver's eye, mouth and ear.

[1]  José Eugenio Naranjo,et al.  Autonomous collision avoidance system based on accurate knowledge of the vehicle surroundings , 2015 .

[2]  Cordelia Schmid,et al.  DeepFlow: Large Displacement Optical Flow with Deep Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Bing-Fei Wu,et al.  Driving behaviour-based event data recorder , 2014 .

[4]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Bailing Zhang,et al.  Recognition of driving postures by contourlet transform and random forests , 2012 .

[6]  Mohan M. Trivedi,et al.  Multi-spectral and multi-perspective video arrays for driver body tracking and activity analysis , 2007, Comput. Vis. Image Underst..

[7]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[8]  Margaret M. Peden,et al.  World Report on Road Traffic Injury Prevention , 2004 .

[9]  Wei Li,et al.  Driver fatigue evaluation model with integration of multi-indicators based on dynamic Bayesian network , 2015 .

[10]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Yuning Jiang,et al.  Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[12]  Tzyy-Ping Jung,et al.  EEG-based drowsiness estimation for safety driving using independent component analysis , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Miguel Ángel Sotelo,et al.  Real-time system for monitoring driver vigilance , 2004, Proceedings of the IEEE International Symposium on Industrial Electronics, 2005. ISIE 2005..

[14]  Jonathan Krause,et al.  Learning Features and Parts for Fine-Grained Recognition , 2014, 2014 22nd International Conference on Pattern Recognition.

[15]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[16]  Shengcai Liao,et al.  Deep Metric Learning for Person Re-identification , 2014, 2014 22nd International Conference on Pattern Recognition.

[17]  Chin-Teng Lin,et al.  Development of Wireless Brain Computer Interface With Embedded Multitask Scheduling and its Application on Real-Time Driver's Drowsiness Detection and Warning , 2008, IEEE Transactions on Biomedical Engineering.

[18]  Satoru Furugori,et al.  Estimation of driver fatigue by pressure distribution on seat in long term driving , 2005 .

[19]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[20]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Zhiwei Zhu,et al.  Real-time nonintrusive monitoring and prediction of driver fatigue , 2004, IEEE Transactions on Vehicular Technology.

[22]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Trevor Darrell,et al.  PANDA: Pose Aligned Networks for Deep Attribute Modeling , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[25]  Paul Watta,et al.  Nonparametric Approaches for Estimating Driver Pose , 2007, IEEE Transactions on Vehicular Technology.

[26]  Paul Stephen Rau,et al.  Drowsy Driver Detection and Warning System for Commercial Vehicle Drivers: Field Operational Test Design, Data Analyses, and Progress , 2005 .

[27]  M Cameron,et al.  World Report on Road Traffic Injury Prevention. , 2004 .

[28]  Luis M. Bergasa,et al.  Real-time system for monitoring driver vigilance , 2005, ISIE 2005.

[29]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[30]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[31]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[32]  Yongzhao Zhan,et al.  Learning Salient Features for Speech Emotion Recognition Using Convolutional Neural Networks , 2014, IEEE Transactions on Multimedia.

[33]  Omer Tsimhoni,et al.  Model-Based Analysis and Classification of Driver Distraction Under Secondary Tasks , 2010, IEEE Transactions on Intelligent Transportation Systems.

[34]  Quoc V. Le,et al.  Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis , 2011, CVPR 2011.