Embedded Fatigue Detection Using Convolutional Neural Networks with Mobile Integration

Fatigued or drowsy drivers pose a significant risk of causing life-threatening accidents. Yet, many sleep-deprived drivers are behind the wheels exposing lives to danger. In this paper, we propose a low-cost and real-time embedded system for fatigue detection using convolutional neural networks (CNN). Our system starts by spatially processing the video signal using a real-time face detection algorithm to establish a region of interest and reduce computations. The video signal comes from a camera module mounted on the car dashboard connected to an embedded Linux board set to monitor the driver's eyes. Detected faces are then passed to an optimized fatigue recognition CNN binary classifier to detect the event of fatigued or normal driving. When temporally persistent fatigue is detected, alerts are sent to the driver's smart phone, and to possibly others, for prevention measures to be taken before accidents happen. Our testing shows that the system can robustly detect fatigue and can effectively be deployed to address the problem.

[1]  Bo Cheng,et al.  Driver drowsiness recognition based on computer vision technology , 2012 .

[2]  Amar Mitiche,et al.  Visual reconstruction of ground plane obstacles in a sparse view robot environment , 2006, Graph. Model..

[3]  Hassan Hajjdiab,et al.  Plant species recognition using leaf contours , 2011, 2011 IEEE International Conference on Imaging Systems and Techniques.

[4]  Ye-Hoon Kim,et al.  Real-Time Driver Drowsiness Detection for Embedded System Using Model Compression of Deep Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Jun-Juh Yan,et al.  Real-Time Driver Drowsiness Detection System Based on PERCLOS and Grayscale Image Processing , 2016, 2016 International Symposium on Computer, Consumer and Control (IS3C).

[6]  Amit Sethi,et al.  Drowsy driver detection using representation learning , 2014, 2014 IEEE International Advance Computing Conference (IACC).

[7]  Mahmood Fathy,et al.  A driver face monitoring system for fatigue and distraction detection , 2013 .

[8]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[9]  Shervin Shirmohammadi,et al.  Driver drowsiness monitoring based on yawning detection , 2011, 2011 IEEE International Instrumentation and Measurement Technology Conference.

[10]  Xiaoyang Tan,et al.  Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients , 2014, Pattern Recognit..

[11]  Laura Astolfi,et al.  Assessment of mental fatigue during car driving by using high resolution EEG activity and neurophysiologic indices , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Fei Pan,et al.  Driver Drowsiness Detection System Based on Feature Representation Learning Using Various Deep Networks , 2016, ACCV Workshops.

[13]  Chin-Teng Lin,et al.  Brain Dynamics in Predicting Driving Fatigue Using a Recurrent Self-Evolving Fuzzy Neural Network , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Zuojin Li,et al.  Online Detection of Driver Fatigue Using Steering Wheel Angles for Real Driving Conditions , 2017, Sensors.

[15]  Aurobinda Routray,et al.  A Vision-Based System for Monitoring the Loss of Attention in Automotive Drivers , 2013, IEEE Transactions on Intelligent Transportation Systems.

[16]  Paul Stephen Rau,et al.  Drowsy Driver Detection and Warning System for Commercial Vehicle Drivers: Field Operational Test Design, Data Analyses, and Progress , 2005 .

[17]  Kim Fung Tsang,et al.  An Accurate ECG-Based Transportation Safety Drowsiness Detection Scheme , 2016, IEEE Transactions on Industrial Informatics.

[18]  H. Cai,et al.  An Experiment to Non-Intrusively Collect Physiological Parameters towards Driver State Detection , 2007 .

[19]  Pam Fischer,et al.  Wake Up Call! Understanding Drowsy Driving and What States Can Do , 2016 .