On-line Driver Distraction Detection using Long Short-Term Memory

Lane-keeping assistance systems for vehicles may be more acceptable to users if the assistance was adaptive to the driver’s state. To adapt systems in this way, a method for detection of driver distraction is needed. Thus, we propose a novel technique for on-line detection of driver’s distraction, modeling the long-range temporal context of driving and head tracking data. We show that Long Short-Term Memory (LSTM) recurrent neural networks enable a reliable, subject-independent detection of inattention with an accuracy of up to 96.6 %. Thereby our LSTM framework significantly outperforms conventional approaches such as Support Vector Machines.

[1]  John D. Lee,et al.  Real-Time Detection of Driver Cognitive Distraction Using Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[2]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[3]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[4]  Thomas A. Dingus,et al.  The 100-Car Naturalistic Driving Study Phase II – Results of the 100-Car Field Experiment , 2006 .

[5]  Björn W. Schuller,et al.  Bidirectional LSTM Networks for Context-Sensitive Keyword Detection in a Cognitive Virtual Agent Framework , 2010, Cognitive Computation.

[6]  Wolfgang Birk,et al.  Evaluation of Lane Departure Warnings for Drowsy Drivers , 2006 .

[7]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[8]  Motoyuki Akamatsu,et al.  Prediction of Human Driving Behavior Using Dynamic Bayesian Networks , 2006, IEICE Trans. Inf. Syst..

[9]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[10]  John D. Lee,et al.  Driver Cognitive Distraction Detection Using Eye Movements , 2008 .

[11]  Michael A. Regan,et al.  Driver distraction: A review of the literature , 2003 .

[12]  Björn W. Schuller,et al.  A multidimensional dynamic time warping algorithm for efficient multimodal fusion of asynchronous data streams , 2009, Neurocomputing.

[13]  Gustav Markkula,et al.  Driver cognitive distraction detection: Feature estimation and implementation , 2007 .

[14]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[15]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[16]  Michael J. Goodman,et al.  NHTSA DRIVER DISTRACTION RESEARCH: PAST, PRESENT, AND FUTURE , 2001 .

[17]  Alex Pentland,et al.  Modeling and Prediction of Human Behavior , 1999, Neural Computation.

[18]  Karel Brookhuis,et al.  The feasibility of detecting phone-use related driver distraction , 2001 .

[19]  Noel Massey,et al.  Detecting driver inattention in the absence of driver monitoring sensors , 2004, 2004 International Conference on Machine Learning and Applications, 2004. Proceedings..

[20]  Berthold Färber,et al.  Driver distraction based lane-keeping assistance , 2009 .

[21]  Harry Zhang,et al.  Identification of Real-Time Diagnostic Measures of Visual Distraction With an Automatic Eye-Tracking System , 2006, Hum. Factors.

[22]  S.P. Hoogendoorn,et al.  Field Operational Test "The Assisted Driver" , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[23]  Björn W. Schuller,et al.  Robust discriminative keyword spotting for emotionally colored spontaneous speech using bidirectional LSTM networks , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[24]  John D. Lee,et al.  Nonintrusive Detection of Driver Cognitive Distraction in Real Time Using Bayesian Networks , 2007 .

[25]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[26]  Yoichi Sugimoto,et al.  Effectiveness Estimation Method for Advanced Driver Assistance System and Its Application to Collision Mitigation Brake System , 2005 .

[27]  Liu Qiao,et al.  Learning algorithm of environmental recognition in driving vehicle , 1995, IEEE Trans. Syst. Man Cybern..

[28]  R. Freymann.  The role of driver assistance systems in a future traffic scenario , 2006, 2006 IEEE International Conference on Control Applications.

[29]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[30]  Björn W. Schuller,et al.  Combining Long Short-Term Memory and Dynamic Bayesian Networks for Incremental Emotion-Sensitive Artificial Listening , 2010, IEEE Journal of Selected Topics in Signal Processing.

[31]  Björn W. Schuller,et al.  OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[32]  Cataldo Guaragnella,et al.  A visual approach for driver inattention detection , 2007, Pattern Recognit..

[33]  Zhiwei Zhu,et al.  Real-time nonintrusive monitoring and prediction of driver fatigue , 2004, IEEE Transactions on Vehicular Technology.

[34]  Qiang Ji,et al.  A probabilistic framework for modeling and real-time monitoring human fatigue , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[35]  Michael J. Goodman,et al.  The role of driver inattention in crashes: new statistics from the 1995 crashworthiness data system , 1996 .

[36]  Yuan Yao,et al.  Combining flat and structured representations for fingerprint classification with recursive neural networks and support vector machines , 2003, Pattern Recognit..