What Can Be Predicted from Six Seconds of Driver Glances?

We consider a large dataset of real-world, on-road driving from a 100-car naturalistic study to explore the predictive power of driver glances and, specifically, to answer the following question: what can be predicted about the state of the driver and the state of the driving environment from a 6-second sequence of macro-glances? The context-based nature of such glances allows for application of supervised learning to the problem of vision-based gaze estimation, making it robust, accurate, and reliable in messy, real-world conditions. So, it's valuable to ask whether such macro-glances can be used to infer behavioral, environmental, and demographic variables? We analyze 27 binary classification problems based on these variables. The takeaway is that glance can be used as part of a multi-sensor real-time system to predict radio-tuning, fatigue state, failure to signal, talking, and several environment variables.

[1]  Yulan Liang,et al.  Detecting driver distraction , 2009 .

[2]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[3]  Alex Fridman,et al.  'Owl' and 'Lizard': patterns of head pose and eye pose in driver gaze classification , 2015, IET Comput. Vis..

[4]  Susan A. Soccolich,et al.  Identification of Cognitive Load in Naturalistic Driving , 2015 .

[5]  John D. Lee,et al.  Real-Time Detection of Driver Cognitive Distraction Using Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[6]  Gerd Wanielik,et al.  Head tracking based glance area estimation for driver behaviour modelling during lane change execution , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[7]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[8]  Mark Vollrath,et al.  Accident Analysis and Prevention , 2009 .

[9]  Albert Kircher,et al.  Comparison of Two Eye-Gaze Based Real-Time Driver Distraction Detection Algorithms in a Small-Scale Field Operational Test , 2017 .

[10]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[11]  Mauricio Muñoz,et al.  Distinguishing patterns in drivers’ visual attention allocation using Hidden Markov Models , 2016 .

[12]  Anna Anund,et al.  Detecting Driver Sleepiness Using Optimized Nonlinear Combinations of Sleepiness Indicators , 2011, IEEE Transactions on Intelligent Transportation Systems.

[13]  Prabir Bhattacharya,et al.  A driver fatigue recognition model based on information fusion and dynamic Bayesian network , 2010, Inf. Sci..

[14]  Shunzheng Yu,et al.  Hidden semi-Markov models , 2010, Artif. Intell..

[15]  David D. Woods,et al.  Cognitive Technologies: The Design of Joint Human-Machine Cognitive Systems , 1986, AI Mag..

[16]  Thomas A. Dingus,et al.  The 100-Car Naturalistic Driving Study Phase II – Results of the 100-Car Field Experiment , 2006 .

[17]  Mohan M. Trivedi,et al.  Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation , 2006, IEEE Transactions on Intelligent Transportation Systems.

[18]  Yu Zhang,et al.  Evaluation of Strategies for Integrated Classification of Visual-Manual and Cognitive Distractions in Driving , 2016, Hum. Factors.

[19]  Mohan M. Trivedi,et al.  Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis , 2013, IEEE Transactions on Intelligent Transportation Systems.

[20]  R Martins,et al.  Eye blinking as an indicator of fatigue and mental load—a systematic review , 2015 .

[21]  Wendy Ju,et al.  Joint Cognition in Automated Driving: Combining Human and Machine Intelligence to Address Novel Problems , 2015, AAAI Spring Symposia.

[22]  Thomas A. Dingus,et al.  The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data , 2006 .

[23]  John D. Lee,et al.  How Dangerous Is Looking Away From the Road? Algorithms Predict Crash Risk From Glance Patterns in Naturalistic Driving , 2012, Hum. Factors.

[24]  Erik Hollnagel,et al.  Cognitive Systems Engineering: New Wine in New Bottles , 1983, Int. J. Man Mach. Stud..

[25]  Johan Engström,et al.  Sensitivity of eye-movement measures to in-vehicle task difficulty , 2005 .

[26]  Cataldo Guaragnella,et al.  A visual approach for driver inattention detection , 2007, Pattern Recognit..

[27]  David G. Kidd,et al.  Multi-modal demands of a smartphone used to place calls and enter addresses during highway driving relative to two embedded systems , 2016, Ergonomics.

[28]  John D. Lee,et al.  Nonintrusive Detection of Driver Cognitive Distraction in Real Time Using Bayesian Networks , 2007 .

[29]  Jingyu Yang,et al.  Driver Fatigue Detection: A Survey , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[30]  Jing Zhang,et al.  Driver cognitive workload estimation: a data-driven perspective , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[31]  A. Mortazavi,et al.  Evaluation of a Smart Algorithm for Commercial Vehicle Driver Drowsiness Detection , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[32]  Matthew J. Johnson,et al.  Bayesian nonparametric hidden semi-Markov models , 2012, J. Mach. Learn. Res..

[33]  Gamini Dissanayake,et al.  Driver Drowsiness Classification Using Fuzzy Wavelet-Packet-Based Feature-Extraction Algorithm , 2011, IEEE Transactions on Biomedical Engineering.

[34]  Benjamin Georgi,et al.  The General Hidden Markov Model Library : Analyzing Systems with Unobservable States , 2004 .

[35]  George S. Fishman,et al.  Discrete-Event Simulation : Modeling, Programming, and Analysis , 2001 .

[36]  Fred L. Mannering,et al.  The statistical analysis of crash-frequency data: A review and assessment of methodological alternatives , 2010 .

[37]  Mica R. Endsley,et al.  Integrating Human Capabilities into Biosurveillance Systems: A Study of Biosurveillance and Situation Awareness , 2010 .