Driver Distraction Using Visual-Based Sensors and Algorithms

Driver distraction, defined as the diversion of attention away from activities critical for safe driving toward a competing activity, is increasingly recognized as a significant source of injuries and fatalities on the roadway. Additionally, the trend towards increasing the use of in-vehicle information systems is critical because they induce visual, biomechanical and cognitive distraction and may affect driving performance in qualitatively different ways. Non-intrusive methods are strongly preferred for monitoring distraction, and vision-based systems have appeared to be attractive for both drivers and researchers. Biomechanical, visual and cognitive distractions are the most commonly detected types in video-based algorithms. Many distraction detection systems only use a single visual cue and therefore, they may be easily disturbed when occlusion or illumination changes appear. Moreover, the combination of these visual cues is a key and challenging aspect in the development of robust distraction detection systems. These visual cues can be extracted mainly by using face monitoring systems but they should be completed with more visual cues (e.g., hands or body information) or even, distraction detection from specific actions (e.g., phone usage). Additionally, these algorithms should be included in an embedded device or system inside a car. This is not a trivial task and several requirements must be taken into account: reliability, real-time performance, low cost, small size, low power consumption, flexibility and short time-to-market. The key points for the development and implementation of sensors to carry out the detection of distraction will also be reviewed. This paper shows a review of the role of computer vision technology applied to the development of monitoring systems to detect distraction. Some key points considered as both future work and challenges ahead yet to be solved will also be addressed.

[1]  Mohan M. Trivedi,et al.  Head Pose Estimation and Augmented Reality Tracking: An Integrated System and Evaluation for Monitoring Driver Awareness , 2010, IEEE Transactions on Intelligent Transportation Systems.

[2]  John D Lee,et al.  Using a Layered Algorithm to Detect Driver Cognitive Distraction , 2017 .

[3]  Haruki Kawanaka,et al.  Effect of pattern recognition features on detection for driver's cognitive distraction , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[4]  R. Gliklich,et al.  Texting while driving: the development and validation of the distracted driving survey and risk score among young adults , 2016, Injury Epidemiology.

[5]  José María Armingol,et al.  Driver drowsiness detection system under infrared illumination for an intelligent vehicle , 2011 .

[6]  Frank Drews,et al.  Text Messaging During Simulated Driving , 2009, Hum. Factors.

[7]  Arturo de la Escalera,et al.  Head pose estimation based on 2D and 3D information , 2014 .

[8]  Slawomir Gruszczynski,et al.  Hybrid computer vision system for drivers' eye recognition and fatigue monitoring , 2014, Neurocomputing.

[9]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[10]  Olga Sourina,et al.  Driver Workload Detection in On-Road Driving Environment Using Machine Learning , 2015 .

[11]  Thomas A. Dingus,et al.  The 100-Car Naturalistic Driving Study Phase II – Results of the 100-Car Field Experiment , 2006 .

[12]  Mohan M. Trivedi,et al.  Toward Privacy-Protecting Safety Systems for Naturalistic Driving Videos , 2014, IEEE Transactions on Intelligent Transportation Systems.

[13]  Christopher A. Monk,et al.  A conceptual framework and taxonomy for understanding and categorizing driver inattention , 2013 .

[14]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[15]  Maytham Safar,et al.  Ambient Technology in Vehicles: The Benefits and Risks , 2016, ANT/SEIT.

[16]  Gang Li,et al.  Detection of Driver Drowsiness Using Wavelet Analysis of Heart Rate Variability and a Support Vector Machine Classifier , 2013, Sensors.

[17]  Mohan M. Trivedi,et al.  Vision on Wheels: Looking at Driver, Vehicle, and Surround for On-Road Maneuver Analysis , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[18]  Mohan M. Trivedi,et al.  Driver hand activity analysis in naturalistic driving studies: challenges, algorithms, and experimental studies , 2013, J. Electronic Imaging.

[19]  Feng Guo,et al.  Keep your eyes on the road: young driver crash risk increases according to duration of distraction. , 2014, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[20]  C. Long,et al.  P-FAD: Real-Time Face Detection Scheme on Embedded Smart Cameras , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[21]  Cary Stothart,et al.  The attentional cost of receiving a cell phone notification. , 2015, Journal of experimental psychology. Human perception and performance.

[22]  Andrew L. Kun,et al.  Estimating cognitive load using pupil diameter during a spoken dialogue task , 2013, AutomotiveUI.

[23]  Robert J Carroll,et al.  The development of a naturalistic data collection system to perform critical incident analysis: an investigation of safety and fatigue issues in long-haul trucking. , 2006, Accident; analysis and prevention.

[24]  Richard A. Young,et al.  Cognitive Distraction While Driving: A Critical Review of Definitions and Prevalence in Crashes , 2012 .

[25]  Christophe Bobda,et al.  A hardware/software prototyping system for driving assistance investigations , 2016, Journal of Real-Time Image Processing.

[26]  Moshe Eizenman,et al.  An on-road assessment of cognitive distraction: impacts on drivers' visual behavior and braking performance. , 2007, Accident; analysis and prevention.

[27]  Fernando García,et al.  Data Fusion for Driver Behaviour Analysis , 2015, Sensors.

[28]  Kiyoung Choi,et al.  SoC Architecture for Automobile Vision System , 2014 .

[29]  Omer Tsimhoni TIME-SHARING OF A VISUAL IN-VEHICLE TASK WHILE DRIVING: FINDINGS FROM THE TASK OCCLUSION METHOD , 2003 .

[30]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Zhi-Hua Zhou,et al.  A literature survey on robust and efficient eye localization in real-life scenarios , 2013, Pattern Recognit..

[32]  B. Reimer,et al.  Physiological Reactivity to Graded Levels of Cognitive Workload across Three Age Groups: An On-Road Evaluation , 2010 .

[33]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Yan Yang,et al.  Cluster Regularized Extreme Learning Machine for Detecting Mixed-Type Distraction in Driving , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[35]  Mohan M. Trivedi,et al.  The Power Is in Your Hands: 3D Analysis of Hand Gestures in Naturalistic Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[36]  Zhiping Lin,et al.  Detection of Drivers’ Distraction Using Semi-Supervised Extreme Learning Machine , 2015 .

[37]  Nadia W. Mullen,et al.  Physiological responses to simulated and on-road driving. , 2011, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[38]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[39]  Bok-Suk Shin,et al.  Visual lane analysis and higher-order tasks: a concise review , 2014, Machine Vision and Applications.

[40]  Johan Engström,et al.  Effects of visual and cognitive load in real and simulated motorway driving , 2005 .

[41]  Aurobinda Routray,et al.  A Vision-Based System for Monitoring the Loss of Attention in Automotive Drivers , 2013, IEEE Transactions on Intelligent Transportation Systems.

[42]  Robert P. Loce,et al.  A machine learning approach for detecting cell phone usage , 2015, Electronic Imaging.

[43]  R. Apparies,et al.  A psychophysiological investigation of the effects of driving longer-combination vehicles. , 1998, Ergonomics.

[44]  Ginés García-Mateos,et al.  Estimating 3D facial pose in video with just three points , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[45]  Oihana Otaegui,et al.  A reconfigurable embedded vision system for advanced driver assistance , 2015, Journal of Real-Time Image Processing.

[46]  Fakhri Karray,et al.  Multi-distributions Particle Filter for Eye Tracking Inside a Vehicle , 2013, ICIAR.

[47]  Adil Haider,et al.  An evidence-based review: Distracted driver , 2015, The journal of trauma and acute care surgery.

[48]  A. Miller,et al.  An optimized vision library approach for embedded systems , 2011, CVPR 2011 WORKSHOPS.

[49]  Frans Coenen,et al.  Driving posture recognition by convolutional neural networks , 2015, 2015 11th International Conference on Natural Computation (ICNC).

[50]  Murtaza Bulut,et al.  Camera-based heart rate monitoring in highly dynamic light conditions , 2013, 2013 International Conference on Connected Vehicles and Expo (ICCVE).

[51]  John D. Lee,et al.  A hybrid Bayesian Network approach to detect driver cognitive distraction , 2014 .

[52]  Marios Savvides,et al.  Driver cell phone usage detection on Strategic Highway Research Program (SHRP2) face view videos , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[53]  T. Victor Keeping Eye and Mind on the Road , 2005 .

[54]  Akira Hattori,et al.  Development of Forward Collision Warning System using the Driver Behavioral Information , 2006 .

[55]  Mubarak Shah,et al.  Determining driver visual attention with one camera , 2003, IEEE Trans. Intell. Transp. Syst..

[56]  Michael J. Goodman,et al.  NHTSA DRIVER DISTRACTION RESEARCH: PAST, PRESENT, AND FUTURE , 2001 .

[57]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[58]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Mohan M. Trivedi,et al.  Snap-DAS: A vision-based driver assistance system on a SnapdragonTM embedded platform , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[61]  Fridtjof Stein The challenge of putting vision algorithms into a car , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[62]  Luke Fletcher,et al.  Correlating driver gaze with the road scene for driver assistance systems , 2005, Robotics Auton. Syst..

[63]  Qiang Ji,et al.  3D Face pose estimation and tracking from a monocular camera , 2002, Image Vis. Comput..

[64]  Fakhri Karray,et al.  Driver distraction detection and recognition using RGB-D sensor , 2015, ArXiv.

[65]  Myounghoon Jeon,et al.  Anger Effects on Driver Situation Awareness and Driving Performance , 2014, PRESENCE: Teleoperators and Virtual Environments.

[66]  Thomas A. Dingus,et al.  Driver Inattention: A Contributing Factor to Crashes and Near-Crashes , 2005 .

[67]  Mohan M. Trivedi,et al.  Predicting driver maneuvers by learning holistic features , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[68]  Marco Dozza,et al.  Driving context and visual-manual phone tasks influence glance behavior in naturalistic driving , 2014 .

[69]  Richard Young,et al.  Revised Odds Ratio Estimates of Secondary Tasks: A Re-Analysis of the 100-Car Naturalistic Driving Study Data , 2015 .

[70]  M A Vidulich,et al.  Performance-based and physiological measures of situational awareness. , 1994, Aviation, space, and environmental medicine.

[71]  Pedro Jiménez,et al.  Analysing Driver's Attention Level using Computer Vision , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[72]  Christopher D. Wickens,et al.  An introduction to human factors engineering , 1997 .

[73]  Hang-Bong Kang,et al.  Various Approaches for Driver and Driving Behavior Monitoring: A Review , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[74]  Oihana Otaegui,et al.  Embedded Platforms for Computer Vision-based Advanced Driver Assistance Systems: a Survey , 2015, ArXiv.

[75]  John D Lee,et al.  Dynamics of Driver Distraction: The process of engaging and disengaging. , 2014, Annals of advances in automotive medicine. Association for the Advancement of Automotive Medicine. Annual Scientific Conference.

[76]  Charlene Hallett,et al.  Driver distraction and driver inattention: definition, relationship and taxonomy. , 2011, Accident; analysis and prevention.

[77]  Hema Swetha Koppula,et al.  Car that Knows Before You Do: Anticipating Maneuvers via Learning Temporal Driving Models , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[78]  Mohan M. Trivedi,et al.  Understanding head and hand activities and coordination in naturalistic driving videos , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[79]  Jiang Yuying,et al.  A surveillance method for driver's fatigue and distraction based on machine vision , 2011, Proceedings 2011 International Conference on Transportation, Mechanical, and Electrical Engineering (TMEE).

[80]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[81]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[82]  Thomas A. Dingus,et al.  The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data , 2006 .

[83]  Thomas A. Dingus,et al.  An overview of the 100-car naturalistic study and findings , 2005 .

[84]  Reinhard Klette,et al.  Look at the Driver, Look at the Road: No Distraction! No Accident! , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[85]  Riad I. Hammoud,et al.  Driver State Monitor from DELPHI , 2005, CVPR.

[86]  Frans Coenen,et al.  Driving Posture Recognition by Joint Application of Motion History Image and Pyramid Histogram of Oriented Gradients , 2013 .

[87]  Roger Reynaud,et al.  Towards an Automatic Prediction of Image Processing Algorithms Performances on Embedded Heterogeneous Architectures , 2015, 2015 44th International Conference on Parallel Processing Workshops.

[88]  Dot Hs,et al.  The 100-Car Naturalistic Driving Study Phase II - Results of the 100-Car Field Experiment , 2006 .

[89]  Jean-Michel Auberlet,et al.  Impact of Narrower Lane Width , 2009 .

[90]  A. Singhal,et al.  The emotional side of cognitive distraction: Implications for road safety. , 2013, Accident; analysis and prevention.

[91]  R. S. Lynch,et al.  Anger, aggression, risky behavior, and crash-related outcomes in three groups of drivers. , 2003, Behaviour research and therapy.

[92]  J. Stack Driving safety. , 1966, The Journal of trauma.

[93]  Björn W. Schuller,et al.  Emotion on the Road - Necessity, Acceptance, and Feasibility of Affective Computing in the Car , 2010, Adv. Hum. Comput. Interact..

[94]  M. A. Recarte,et al.  Mental workload while driving: effects on visual search, discrimination, and decision making. , 2003, Journal of experimental psychology. Applied.

[95]  Jovitha Jerome,et al.  Embedded implementation of facial landmarks detection using extended active shape model approach , 2014, 2014 International Conference on Embedded Systems (ICES).

[96]  Qiang Ji,et al.  Real Time Visual Cues Extraction for Monitoring Driver Vigilance , 2001, ICVS.

[97]  Seong G. Kong,et al.  Visual Analysis of Eye State and Head Pose for Driver Alertness Monitoring , 2013, IEEE Transactions on Intelligent Transportation Systems.

[98]  Haruki Kawanaka,et al.  Driver's cognitive distraction detection using physiological features by the adaboost , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[99]  Enrico Grosso,et al.  Real time detection of driver attention: Emerging solutions based on robust iconic classifiers and dictionary of poses , 2014 .

[100]  Reinhard Klette,et al.  Adaptive Haar-like classifier for eye status detection under non-ideal lighting conditions , 2012, IVCNZ '12.

[101]  Oihana Otaegui,et al.  On creating vision-based advanced driver assistance systems , 2015 .

[102]  Erika E. Miller,et al.  Effects of Roadway on Driver Stress: An On-Road Study using Physiological Measures , 2013 .

[103]  Eric Armengaud,et al.  Filling the gap between automotive systems, safety, and software engineering , 2015, Elektrotech. Informationstechnik.

[104]  Mohan M. Trivedi,et al.  Drive quality analysis of lane change maneuvers for naturalistic driving studies , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[105]  Harini Veeraraghavan,et al.  DETECTING DRIVER FATIGUE THROUGH THE USE OF ADVANCED FACE MONITORING TECHNIQUES , 2001 .

[106]  Wen-Hung Chao,et al.  The short-time fractal scaling of heart rate variability to estimate the mental stress of driver , 2004, IEEE International Conference on Networking, Sensing and Control, 2004.

[107]  Narciso García Santos,et al.  Video-based Driver Assistance Systems , 2008 .

[108]  Heng Yang,et al.  Facial feature point detection: A comprehensive survey , 2014, Neurocomputing.

[109]  Monica G Lichty,et al.  Towards Operationalizing Driver Distraction , 2013 .

[110]  Bailing Zhang,et al.  Vision-based Classification of Driving Postures by Efficient Feature Extraction and Bayesian Approach , 2013, J. Intell. Robotic Syst..

[111]  Pedro Boloto Chambino Android-based implementation of Eulerian Video Magnification for vital signs monitoring , 2013 .

[112]  Jing Zhang,et al.  Driver cognitive workload estimation: a data-driven perspective , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[113]  Björn W. Schuller,et al.  Online Driver Distraction Detection Using Long Short-Term Memory , 2011, IEEE Transactions on Intelligent Transportation Systems.

[114]  Mohan M. Trivedi,et al.  On the design and evaluation of robust head pose for visual user interfaces: algorithms, databases, and comparisons , 2012, AutomotiveUI.

[115]  Fernando De la Torre,et al.  Driver Gaze Tracking and Eyes Off the Road Detection System , 2015, IEEE Transactions on Intelligent Transportation Systems.

[116]  Yuying Jiang,et al.  Driver Sleepiness Detection System Based on Eye Movements Variables , 2013 .

[117]  Shuyan Zhao,et al.  An Automatic Face Recognition System in the Near Infrared Spectrum , 2005, MLDM.

[118]  Yulan Liang,et al.  Detecting driver distraction , 2009 .

[119]  Fei Wang,et al.  A FPGA based driver drowsiness detecting system , 2005, IEEE International Conference on Vehicular Electronics and Safety, 2005..

[120]  Miguel Torres-Torriti,et al.  Optical Flow and Driver's Kinematics Analysis for State of Alert Sensing , 2013, Sensors.

[121]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[122]  Nanning Zheng,et al.  Visual recognition of driver hand-held cell phone use based on hidden CRF , 2011, Proceedings of 2011 IEEE International Conference on Vehicular Electronics and Safety.

[123]  Gary Burnett,et al.  Defining Driver Distraction , 2005 .

[124]  Jean-Michel Auberlet,et al.  The impact of perceptual treatments on driver's behavior: from driving simulator studies to field tests--first results. , 2012, Accident; analysis and prevention.

[125]  Dot Hs An Analysis of Driver Inattention Using a Case-Crossover Approach On 100-Car Data: Final Report , 2010 .

[126]  Ricardo Sanz,et al.  Embedded Intelligence on Chip: Some FPGAbased Design Experiences , 2010 .

[127]  Jorge Batista,et al.  A Real-Time Driver Visual Attention Monitoring System , 2005, IbPRIA.

[128]  A. Fort,et al.  Inattention behind the wheel: how factual internal thoughts impact attentional control while driving , 2014 .

[129]  Mark Asbridge,et al.  A meta-analysis of the effects of texting on driving. , 2014, Accident; analysis and prevention.

[130]  Mohan M. Trivedi,et al.  Head, Eye, and Hand Patterns for Driver Activity Recognition , 2014, 2014 22nd International Conference on Pattern Recognition.

[131]  A. Baddeley Selective attention and performance in dangerous environments. , 1972, British journal of psychology.

[132]  Alexander Zelinsky,et al.  Vision In and Out of Vehicles: Integrated Driver and Road Scene Monitoring , 2002, ISER.

[133]  Yan Yang The effects of increased workload on driving performance and visual behaviour , 2011 .

[134]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[135]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[136]  Miguel Ángel Sotelo,et al.  Real-time system for monitoring driver vigilance , 2004, Proceedings of the IEEE International Symposium on Industrial Electronics, 2005. ISIE 2005..

[137]  Elizabeth W. Dunn,et al.  "Silence Your Phones": Smartphone Notifications Increase Inattention and Hyperactivity Symptoms , 2016, CHI.

[138]  Ranga Rodrigo,et al.  FPGA-based compact and flexible architecture for real-time embedded vision systems , 2009, 2009 International Conference on Industrial and Information Systems (ICIIS).

[139]  Moshe Eizenman,et al.  THE IMPACT OF COGNITIVE DISTRACTION ON DRIVER VISUAL BEHAVIOUR AND VEHICLE CONTROL , 2002 .

[140]  David Crundall,et al.  Driver's visual attention as a function of driving experience and visibility. Using a driving simulator to explore drivers' eye movements in day, night and rain driving. , 2010, Accident; analysis and prevention.

[141]  Fanglin Chen,et al.  CarSafe app: alerting drowsy and distracted drivers using dual cameras on smartphones , 2013, MobiSys.

[142]  Miguel Torres-Torriti,et al.  Face salient points and eyes tracking for robust drowsiness detection , 2012, Robotica.

[143]  John D. Lee,et al.  Comparing Support Vector Machines ( SVMs ) and Bayesian Networks ( BNs ) in detecting driver cognitive distraction using eye movements , 2007 .

[144]  Rubén Usamentiaga,et al.  Unobtrusive health monitoring system using video-based physiological information and activity measurements , 2015, 2015 International Conference on Computer, Information and Telecommunication Systems (CITS).

[145]  Karlene K. Ball,et al.  Evaluating the Driving Ability of Older Adults , 1994 .

[146]  Ole Helvig Jensen,et al.  Implementing the Viola-Jones Face Detection Algorithm , 2008 .

[147]  L. Mulder Measurement and analysis methods of heart rate and respiration for use in applied environments , 1992, Biological Psychology.

[148]  Michael J. Jones,et al.  Fully automatic pose-invariant face recognition via 3D pose normalization , 2011, 2011 International Conference on Computer Vision.

[149]  Luis Miguel Bergasa,et al.  Fusion of Optimized Indicators from Advanced Driver Assistance Systems (ADAS) for Driver Drowsiness Detection , 2014, Sensors.

[150]  Zhiwei Zhu,et al.  Real time and non-intrusive driver fatigue monitoring , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[151]  John H. L. Hansen,et al.  Computer Vision Systems for “Context-Aware” Active Vehicle Safety and Driver Assistance , 2009 .

[152]  Chongxun Zheng,et al.  Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. , 2012, Accident; analysis and prevention.

[153]  Jason S. McCarley,et al.  Identifying Mind-wandering behind the Wheel , 2009 .

[154]  Arturo de la Escalera,et al.  Real-Time Warning System for Driver Drowsiness Detection Using Visual Information , 2010, J. Intell. Robotic Syst..

[155]  W. Hernandez,et al.  A low-cost real-time FPGA solution for driver drowsiness detection , 2003, IECON'03. 29th Annual Conference of the IEEE Industrial Electronics Society (IEEE Cat. No.03CH37468).

[156]  Mohan M. Trivedi,et al.  On surveillance for safety critical events: In-vehicle video networks for predictive driver assistance systems , 2015, Comput. Vis. Image Underst..

[157]  Anthony Singhal,et al.  Emotion matters: Implications for distracted driving , 2015 .

[158]  Bin Yang,et al.  Camera-based drowsiness reference for driver state classification under real driving conditions , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[159]  Björn W. Schuller,et al.  On the Necessity and Feasibility of Detecting a Driver's Emotional State While Driving , 2007, ACII.

[160]  Igor S. Pandzic,et al.  A method for object detection based on pixel intensity comparisons , 2013, ArXiv.

[161]  Adel Said Elmaghraby,et al.  A Comparison of Face Detection Algorithms in Visible and Thermal Spectrums , 2012 .

[162]  Rama Chellappa,et al.  A deep pyramid Deformable Part Model for face detection , 2015, 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[163]  Katja Kircher,et al.  Evaluation of methods for the assessment of minimum required attention , 2015 .

[164]  T. Dingus,et al.  Distracted driving and risk of road crashes among novice and experienced drivers. , 2014, The New England journal of medicine.

[165]  Mohan M. Trivedi,et al.  In-vehicle hand activity recognition using integration of regions , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[166]  Susan A. Soccolich,et al.  Identification of Cognitive Load in Naturalistic Driving , 2015 .

[167]  Laura Marie Toole,et al.  Crash Risk and Mobile Device Use Based on Fatigue and Drowsiness Factors in Truck Drivers , 2013 .

[168]  Carlos Hitoshi Morimoto,et al.  Pupil detection and tracking using multiple light sources , 2000, Image Vis. Comput..

[169]  Riad I. Hammoud,et al.  Efficient Real-Time Algorithms for Eye State and Head Pose Tracking in Advanced Driver Support Systems , 2005, CVPR.

[170]  Albert Kircher,et al.  Comparison of Two Eye-Gaze Based Real-Time Driver Distraction Detection Algorithms in a Small-Scale Field Operational Test , 2017 .

[171]  Reinhard Klette,et al.  Simultaneous analysis of driver behaviour and road condition for driver distraction detection , 2011 .

[172]  Bailing Zhang,et al.  Recognition of driving postures by contourlet transform and random forests , 2012 .

[173]  Zhiwei Zhu,et al.  Active facial tracking for fatigue detection , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[174]  J. May,et al.  Eye movement indices of mental workload. , 1990, Acta psychologica.

[175]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[176]  Mikael B. Skov,et al.  Evaluating driver attention and driving behaviour: comparing controlled driving and simulated driving , 2008, BCS HCI.

[177]  Gang Zhou,et al.  Determining driver phone use leveraging smartphone sensors , 2015, Multimedia Tools and Applications.

[178]  Jing Zhang,et al.  Safe Interaction for Drivers: A Review of Driver Distraction Guidelines and Design Implications , 2015 .

[179]  Mohan M. Trivedi,et al.  Looking-in and looking-out vision for Urban Intelligent Assistance: Estimation of driver attentive state and dynamic surround for safe merging and braking , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[180]  Lars Petersson,et al.  Vision in and out of Vehicles , 2003, IEEE Intell. Syst..

[181]  Martti Juhola,et al.  The Classification of Valid and Invalid Beats of Three-Dimensional Nystagmus Eye Movement Signals Using Machine Learning Methods , 2013, Adv. Artif. Neural Syst..

[182]  Takatsugu Hirayama,et al.  Analysis of Temporal Relationships between Eye Gaze and Peripheral Vehicle Behavior for Detecting Driver Distraction , 2013 .

[183]  A Belger,et al.  Effects of alcohol and chronic aspartame ingestion upon performance in aviation relevant cognitive tasks. , 1994, Aviation, space, and environmental medicine.

[184]  Mohan M. Trivedi,et al.  Drive Analysis Using Vehicle Dynamics and Vision-Based Lane Semantics , 2015, IEEE Transactions on Intelligent Transportation Systems.

[185]  Carlos Busso,et al.  Analysis of facial features of drivers under cognitive and visual distractions , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[186]  Albert Kircher,et al.  Distraction and drowsiness – a field study , 2009 .

[187]  Gail A Wainwright,et al.  Distracted Driving and Implications for Injury Prevention in Adults , 2013, Journal of trauma nursing : the official journal of the Society of Trauma Nurses.

[188]  Maja Pantic,et al.  Facial Expression Recognition , 2009, Encyclopedia of Biometrics.

[189]  Riender Happee,et al.  Advantages and Disadvantages of Driving Simulators: A Discussion , 2012 .

[190]  R L Helmreich,et al.  How effective is cockpit resource management training? Exploring issues in evaluating the impact of programs to enhance crew coordination. , 1990, Flight safety digest.

[191]  O Carsten,et al.  Protective or not? [visual distraction] , 2015 .

[192]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[193]  Muhammad Reza Pourshahabi,et al.  A Review on Driver Face Monitoring Systems for Fatigue and Distraction Detection , 2014 .

[194]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[195]  Mohan M. Trivedi,et al.  Looking at Humans in the Age of Self-Driving and Highly Automated Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[196]  Michele Nappi,et al.  Robust Face Recognition for Uncontrolled Pose and Illumination Changes , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[197]  Kang Ryoung Park,et al.  Real-Time Gaze Estimator Based on Driver's Head Orientation for Forward Collision Warning System , 2011, IEEE Transactions on Intelligent Transportation Systems.

[198]  Yajie Tian,et al.  Handbook of face recognition , 2003 .

[199]  Behnoosh Hariri,et al.  YawDD: a yawning detection dataset , 2014, MMSys '14.

[200]  Louis Tijerina,et al.  Driver Workload Metrics Task 2 Final Report , 2006 .

[201]  Stefan Lee,et al.  This Hand Is My Hand: A Probabilistic Approach to Hand Disambiguation in Egocentric Video , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[202]  Mohan M. Trivedi,et al.  Continuous Head Movement Estimator for Driver Assistance: Issues, Algorithms, and On-Road Evaluations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[203]  Mohamad Hoseyn Sigari Driver Hypo-vigilance Detection Based on Eyelid Behavior , 2009, 2009 Seventh International Conference on Advances in Pattern Recognition.

[204]  Dotan Knaan,et al.  Single image face orientation and gaze detection , 2009, Machine Vision and Applications.

[205]  Capers Jones,et al.  Embedded Software: Facts, Figures, and Future , 2009, Computer.

[206]  Arturo de la Escalera,et al.  Driver Drowsiness Warning System Using Visual Information for Both Diurnal and Nocturnal Illumination Conditions , 2010, EURASIP J. Adv. Signal Process..

[207]  John D. Lee,et al.  Real-Time Detection of Driver Cognitive Distraction Using Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[208]  Rafael A. Berri,et al.  A pattern recognition system for detecting use of mobile phones while driving , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[209]  P. Sudhakar Rao,et al.  A Real Time Improved Driver Fatigue Monitoring System , 2014 .

[210]  Shan Bao,et al.  Driver distraction from cell phone use and potential for self-limiting behavior , 2012 .

[211]  Yan Yang,et al.  Driver Distraction Detection Using Semi-Supervised Machine Learning , 2016, IEEE Transactions on Intelligent Transportation Systems.

[212]  Mohan M. Trivedi,et al.  Automatic Critical Event Extraction and Semantic Interpretation by Looking-Inside , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[213]  Mohan M. Trivedi,et al.  Robust and continuous estimation of driver gaze zone by dynamic analysis of multiple face videos , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[214]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[215]  Tania Dukic,et al.  Comparison of eye tracking systems with one and three cameras , 2010, MB '10.

[216]  Céline Craye,et al.  A Multi-Modal Driver Fatigue and Distraction Assessment System , 2015, International Journal of Intelligent Transportation Systems Research.

[217]  Václav Hlavác,et al.  Facial Landmark Tracking by Tree-Based Deformable Part Model Based Detector , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[218]  Albert Kircher,et al.  A Gaze-Based Driver Distraction Warning System and Its Effect on Visual Behavior , 2013, IEEE Transactions on Intelligent Transportation Systems.

[219]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[220]  Václav Hlavác,et al.  Real-time multi-view facial landmark detector learned by the structured output SVM , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[221]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[222]  Amnon Shashua,et al.  A Computer Vision System on a Chip: a case study from the automotive domain , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[223]  K. Oguri,et al.  Driver’s cognitive distraction detection using AdaBoost on pattern recognition basis , 2008, 2008 IEEE International Conference on Vehicular Electronics and Safety.

[224]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[225]  Orhan Bulan,et al.  Driver Cell Phone Usage Detection from HOV/HOT NIR Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[226]  Bryan Reimer,et al.  Classifying driver workload using physiological and driving performance data: two field studies , 2014, CHI.

[227]  Wang Yu,et al.  Head Pose Estimation Based on Head Tracking and the Kalman Filter , 2011 .

[228]  Hiroshi Ishiguro,et al.  DEVELOPMENT OF FACIAL-DIRECTION DETECTION SENSOR , 2006 .

[229]  Albrecht Schmidt,et al.  A Model Relating Pupil Diameter to Mental Workload and Lighting Conditions , 2016, CHI.

[230]  Andrew Morris,et al.  Exploring inattention and distraction in the SafetyNet Accident Causation Database. , 2013, Accident; analysis and prevention.

[231]  Linda Ng Boyle,et al.  Safety implications of providing real-time feedback to distracted drivers. , 2007, Accident; analysis and prevention.

[232]  Bo Li,et al.  P-FAD: Real-time face detection scheme on embedded smart cameras , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[233]  Shuyan Zhao,et al.  Robust Eye Detection under Active Infrared Illumination , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[234]  Kwang Suk Park,et al.  Heart rate estimation from facial photoplethysmography during dynamic illuminance changes , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[235]  Bradley Malin,et al.  Preserving privacy by de-identifying face images , 2005, IEEE Transactions on Knowledge and Data Engineering.

[236]  D. Strayer,et al.  Cell phone-induced failures of visual attention during simulated driving. , 2003, Journal of experimental psychology. Applied.

[237]  Jason S. McCarley,et al.  Mind Wandering Behind the Wheel , 2011, Hum. Factors.

[238]  P. Ekman Pictures of Facial Affect , 1976 .

[239]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[240]  Lena Nilsson,et al.  Effects of cognitive and visual load in real and simulated driving , 2006 .

[241]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[242]  Fernando García,et al.  Driver Monitoring Based on Low-Cost 3-D Sensors , 2014, IEEE Transactions on Intelligent Transportation Systems.

[243]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[244]  Diane Nahl,et al.  Road Rage and Aggressive Driving: Steering Clear of Highway Warfare , 2000 .

[245]  Yongsheng Gao,et al.  Recognition of driving postures by multiwavelet transform and multilayer perceptron classifier , 2012, Eng. Appl. Artif. Intell..

[246]  M. A. Recarte,et al.  Effects of verbal and spatial-imagery tasks on eye fixations while driving. , 2000, Journal of experimental psychology. Applied.

[247]  Fakhri Karray,et al.  A Visual-Based Driver Distraction Recognition and Detection Using Random Forest , 2014, ICIAR.

[248]  Wolfgang Birk,et al.  A driver-distraction-based lane-keeping assistance system , 2007 .

[249]  Linda Ng Boyle,et al.  Mitigating driver distraction with retrospective and concurrent feedback. , 2008, Accident; analysis and prevention.

[250]  Jorge Batista,et al.  A Drowsiness and Point of Attention Monitoring System for Driver Vigilance , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[251]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[252]  Hampton C. Gabler,et al.  Methodology for identifying car following events from naturalistic data , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[253]  John D. Lee,et al.  Nonintrusive Detection of Driver Cognitive Distraction in Real Time Using Bayesian Networks , 2007 .

[254]  Frank Forster Heterogeneous Processors for Advanced Driver Assistance Systems , 2014 .

[255]  Michel F. Sultan,et al.  Monitoring Driver Physiological Parameters for Improved Safety , 2006 .

[256]  Herb M Simpson,et al.  On-road and simulated driving: concurrent and discriminant validation. , 2011, Journal of safety research.

[257]  Mikael B. Skov,et al.  Evaluating Driver Attention and Driving Behavior: Comparing Controlled Driving and Simulated Driving , 2008 .

[258]  John D. Lee,et al.  Detection of Driver Distraction Using Vision-Based Algorithms , 2013 .

[259]  Mohan M. Trivedi,et al.  The rhythms of head, eyes and hands at intersections , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[260]  Mohan M. Trivedi,et al.  Balancing Privacy and Safety: Protecting Driver Identity in Naturalistic Driving Video Data , 2014, AutomotiveUI.

[261]  Amit Konar,et al.  Emotion Recognition: A Pattern Analysis Approach , 2015 .

[262]  Mohan M. Trivedi,et al.  Head Pose Estimation for Driver Assistance Systems: A Robust Algorithm and Experimental Evaluation , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[263]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[264]  Kenneth Sundaraj,et al.  Detecting Driver Drowsiness Based on Sensors: A Review , 2012, Sensors.

[265]  Johan Engström,et al.  Sensitivity of eye-movement measures to in-vehicle task difficulty , 2005 .

[266]  D. Kahneman,et al.  Pupillary, heart rate, and skin resistance changes during a mental task. , 1969, Journal of experimental psychology.

[267]  Miguel Ángel Sotelo,et al.  Real-time robust face tracking for driver monitoring , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[268]  Yorgos Goletsis,et al.  Emotion Recognition in Car Industry , 2015 .

[269]  Cristy Ho,et al.  To What Extent do the Findings of Laboratory-Based Spatial Attention Research Apply to the Real-World Setting of Driving? , 2014, IEEE Transactions on Human-Machine Systems.

[270]  Albert Kircher,et al.  Results of a field study on a driver distraction warning system , 2009 .

[271]  Carlos Busso,et al.  Predicting Perceived Visual and Cognitive Distractions of Drivers With Multimodal Features , 2015, IEEE Transactions on Intelligent Transportation Systems.

[272]  Matthias Mielke,et al.  ASIC implementation of a Gaussian Pyramid for use in autonomous mobile robotics , 2011, 2011 IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS).

[273]  Cataldo Guaragnella,et al.  A visual approach for driver inattention detection , 2007, Pattern Recognit..