Driver Lane Change Intention Inference for Intelligent Vehicles: Framework, Survey, and Challenges

Intelligent vehicles and advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context, as well as the driver status since ADAS share the vehicle control authorities with the human driver. This paper provides an overview of the ego-vehicle driver intention inference (DII), which mainly focuses on the lane change intention on highways. First, a human intention mechanism is discussed in the beginning to gain an overall understanding of the driver intention. Next, the ego-vehicle driver intention is classified into different categories based on various criteria. A complete DII system can be separated into different modules, which consist of traffic context awareness, driver states monitoring, and the vehicle dynamic measurement module. The relationship between these modules and the corresponding impacts on the DII are analyzed. Then, the lane change intention inference system is reviewed from the perspective of input signals, algorithms, and evaluation. Finally, future concerns and emerging trends in this area are highlighted.

[1]  Dario D. Salvucci,et al.  The time course of a lane change: Driver control and eye-movement behavior , 2002 .

[2]  Keiichi Yamada,et al.  Study on driver-intent estimation at yellow traffic signal by using driving simulator , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[3]  Mathias Perrollaz,et al.  Learning-based approach for online lane change intention prediction , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[4]  Takayoshi Kamada,et al.  Classification of Driver Steering Intentions Using an Electroencephalogram , 2008 .

[5]  Fei-Yue Wang,et al.  A Security and Privacy Review of VANETs , 2015, IEEE Transactions on Intelligent Transportation Systems.

[6]  Jianqiang Wang,et al.  Driver intention recognition method based on comprehensive lane-change environment assessment , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[7]  Katja Kircher,et al.  Driver distraction : a review of the literature , 2007 .

[8]  Martin Buss,et al.  Interactive scene prediction for automotive applications , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[9]  Minho Lee,et al.  Driver's lane-change intent identification based on pupillary variation , 2014, 2014 IEEE International Conference on Consumer Electronics (ICCE).

[10]  Petros A. Ioannou,et al.  Personalized Driver/Vehicle Lane Change Models for ADAS , 2015, IEEE Transactions on Vehicular Technology.

[11]  Neville A. Stanton,et al.  Takeover Time in Highly Automated Vehicles: Noncritical Transitions to and From Manual Control , 2017, Hum. Factors.

[12]  Wang Feiyue Concepts and Frameworks of Artificial Transportation Systems , 2004 .

[13]  Wang Feiyue,et al.  Parallel system methods for management and control of complex systems , 2004 .

[14]  Alex Pentland,et al.  Graphical models for driver behavior recognition in a SmartCar , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[15]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[16]  Ioannis Iossifidis,et al.  Towards a Driver Model: Preliminary Study of Lane Change Behavior , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[17]  Johan Wideberg,et al.  Deriving origin destination data from a mobile phone network , 2007 .

[18]  Gerd Wanielik,et al.  Head tracking based glance area estimation for driver behaviour modelling during lane change execution , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[19]  Firas Lethaus,et al.  Windows of Driver Gaze Data: How Early and How Much for Robust Predictions of Driver Intent? , 2013, ICANNGA.

[20]  Jong-Hwan Kim,et al.  Behavior Hierarchy-Based Affordance Map for Recognition of Human Intention and Its Application to Human–Robot Interaction , 2016, IEEE Transactions on Human-Machine Systems.

[21]  Mohan M. Trivedi,et al.  Lane Change Intent Analysis Using Robust Operators and Sparse Bayesian Learning , 2005, IEEE Transactions on Intelligent Transportation Systems.

[22]  Daniel P. Kennedy,et al.  Atypical Visual Saliency in Autism Spectrum Disorder Quantified through Model-Based Eye Tracking , 2015, Neuron.

[23]  Dongpu Cao,et al.  Identification and Analysis of Driver Postures for In-Vehicle Driving Activities and Secondary Tasks Recognition , 2018, IEEE Transactions on Computational Social Systems.

[24]  Firas Lethaus,et al.  A comparison of selected simple supervised learning algorithms to predict driver intent based on gaze data , 2013, Neurocomputing.

[25]  Keqiang Li,et al.  Lane changing intention recognition based on speech recognition models , 2016 .

[26]  Leo Gugerty Situational Awareness in Driving , 2011 .

[27]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Elizabeth Bellis,et al.  National Motor Vehicle Crash Causation Survey (NMVCCS) SAS Analytical Users Manual , 2008 .

[29]  Elisheva Bonchek-Dokow Cognitive Modeling of Human Intention Recognition , 2011 .

[30]  Hermann Winner,et al.  Handbook of Driver Assistance Systems: Basic Information, Components and Systems for Active Safety and Comfort , 2015 .

[31]  I. Ajzen The theory of planned behavior , 1991 .

[32]  Meng Lu,et al.  Driver Intention Recognition Method Using Continuous Hidden Markov Model , 2011, Int. J. Comput. Intell. Syst..

[33]  R. K. Tiwari Design Approach For EEG-Based Human Computer Interaction Driver Monitoring System , 2014 .

[34]  Yili Liu,et al.  Detecting Driver Normal and Emergency Lane-Changing Intentions With Queuing Network-Based Driver Models , 2015, Int. J. Hum. Comput. Interact..

[35]  Erhardt Barth,et al.  Accurate Eye Centre Localisation by Means of Gradients , 2011, VISAPP.

[36]  Bao-Liang Lu,et al.  Emotional state classification from EEG data using machine learning approach , 2014, Neurocomputing.

[37]  Stewart Worrall,et al.  A Recurrent Neural Network Solution for Predicting Driver Intention at Unsignalized Intersections , 2018, IEEE Robotics and Automation Letters.

[38]  Kazuhiro Kosuge,et al.  Dance Step Estimation Method Based on HMM for Dance Partner Robot , 2007, IEEE Transactions on Industrial Electronics.

[39]  John A. Michon,et al.  A critical view of driver behavior models: What do we know , 1985 .

[40]  Peter Carruthers,et al.  The illusion of conscious will , 2007, Synthese.

[41]  Huijing Zhao,et al.  On-Road Vehicle Trajectory Collection and Scene-Based Lane Change Analysis: Part II , 2017, IEEE Trans. Intell. Transp. Syst..

[42]  Jason S. McCarley,et al.  Lane Keeping Under Cognitive Load , 2014, Hum. Factors.

[43]  Y. Sakaguchi,et al.  Prediction of Driving Behavior through Probabilistic Inference , 2003 .

[44]  Ravi Teja Chadalavada,et al.  That's on my mind! robot to human intention communication through on-board projection on shared floor space , 2015, 2015 European Conference on Mobile Robots (ECMR).

[45]  Wei Zhan,et al.  Generic Probabilistic Interactive Situation Recognition and Prediction: From Virtual to Real , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[46]  Christian Laugier,et al.  Context-based estimation of driver intent at road intersections , 2011, 2011 IEEE Symposium on Computational Intelligence in Vehicles and Transportation Systems (CIVTS) Proceedings.

[47]  Ruzena Bajcsy,et al.  Identifying Modes of Intent from Driver Behaviors in Dynamic Environments , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[48]  Mohan M. Trivedi,et al.  Driver Behavior and Situation Aware Brake Assistance for Intelligent Vehicles , 2007, Proceedings of the IEEE.

[49]  Karim A. Tahboub,et al.  Journal of Intelligent and Robotic Systems (2005) DOI: 10.1007/s10846-005-9018-0 Intelligent Human–Machine Interaction Based on Dynamic Bayesian Networks Probabilistic Intention Recognition , 2004 .

[50]  Charlene Hallett,et al.  Driver distraction and driver inattention: definition, relationship and taxonomy. , 2011, Accident; analysis and prevention.

[51]  Takashi Imamura,et al.  Real-time implementation of estimation method for driver's intention on a driving simulator , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[52]  Fei-Yue Wang,et al.  Parallel Control and Management for Intelligent Transportation Systems: Concepts, Architectures, and Applications , 2010, IEEE Transactions on Intelligent Transportation Systems.

[53]  Ji Hyun Yang,et al.  Takeover Requests in Simulated Partially Autonomous Vehicles Considering Human Factors , 2017, IEEE Transactions on Human-Machine Systems.

[54]  René van Paassen,et al.  Inferring the driver's lane change intention using context-based dynamic Bayesian networks , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[55]  Mohan M. Trivedi,et al.  How Would Surround Vehicles Move? A Unified Framework for Maneuver Classification and Motion Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[56]  Michael E. Bratman,et al.  Intention, Plans, and Practical Reason , 1991 .

[57]  Minho Lee,et al.  Identification of human implicit visual search intention based on eye movement and pupillary analysis , 2014, User Modeling and User-Adapted Interaction.

[58]  David Shinar,et al.  Looks Are (Almost) Everything: Where Drivers Look to Get Information , 2008, Hum. Factors.

[59]  Maciej Marcin Michalek,et al.  The Concept of Passive Control Assistance for Docking Maneuvers With N-Trailer Vehicles , 2015, IEEE/ASME Transactions on Mechatronics.

[60]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[61]  Rafael Toledo-Moreo,et al.  IMM-Based Lane-Change Prediction in Highways With Low-Cost GPS/INS , 2009, IEEE Transactions on Intelligent Transportation Systems.

[62]  Wei Liu,et al.  Parking Like a Human: A Direct Trajectory Planning Solution , 2017, IEEE Transactions on Intelligent Transportation Systems.

[63]  Ying Wang,et al.  Detection of Driver Cognitive Distraction: A Comparison Study of Stop-Controlled Intersection and Speed-Limited Highway , 2016, IEEE Transactions on Intelligent Transportation Systems.

[64]  Wolfgang Rosenstiel,et al.  Object-Oriented Bayesian Networks for Detection of Lane Change Maneuvers , 2012, IEEE Intelligent Transportation Systems Magazine.

[65]  Mohan M. Trivedi,et al.  Predicting driver maneuvers by learning holistic features , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[66]  Hiren M. Mandalia,et al.  Using Support Vector Machines for Lane-Change Detection , 2005 .

[67]  H. Takahashi,et al.  A study on mental model for inferring driver's intention , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[68]  Hema Swetha Koppula,et al.  Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture , 2016, ArXiv.

[69]  Hema Swetha Koppula,et al.  Recurrent Neural Networks for driver activity anticipation via sensory-fusion architecture , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[70]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[71]  Azim Eskandarian,et al.  Research advances in intelligent collision avoidance and adaptive cruise control , 2003, IEEE Trans. Intell. Transp. Syst..

[72]  Li Liu,et al.  Driver behavior analysis based on Bayesian network and multiple classifiers , 2010, 2010 IEEE International Conference on Intelligent Computing and Intelligent Systems.

[73]  A. Pentland,et al.  Towards real-time recognition of driver intentions , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[74]  Andreas Lawitzky,et al.  A Game-Theoretic Approach to Replanning-Aware Interactive Scene Prediction and Planning , 2016, IEEE Transactions on Vehicular Technology.

[75]  Minho Lee,et al.  Human intention recognition based on eyeball movement pattern and pupil size variation , 2014, Neurocomputing.

[76]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[77]  Hussein Zedan,et al.  Context-Aware Driver Behavior Detection System in Intelligent Transportation Systems , 2013, IEEE Transactions on Vehicular Technology.

[78]  Ricardo Chavarriaga,et al.  Detection of anticipatory brain potentials during car driving , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[79]  Dongpu Cao,et al.  Hybrid-Learning-Based Classification and Quantitative Inference of Driver Braking Intensity of an Electrified Vehicle , 2018, IEEE Transactions on Vehicular Technology.

[80]  Dario D. Salvucci Modeling Driver Behavior in a Cognitive Architecture , 2006, Hum. Factors.

[81]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[82]  Mohan M. Trivedi,et al.  On surveillance for safety critical events: In-vehicle video networks for predictive driver assistance systems , 2015, Comput. Vis. Image Underst..

[83]  Mohan Manubhai Trivedi,et al.  Dynamics of Driver's Gaze: Explorations in Behavior Modeling and Maneuver Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[84]  Hong Liu,et al.  End-to-End Driving Activities and Secondary Tasks Recognition Using Deep Convolutional Neural Network and Transfer Learning , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[85]  Bo Gao,et al.  Driving Style Recognition for Intelligent Vehicle Control and Advanced Driver Assistance: A Survey , 2018, IEEE Transactions on Intelligent Transportation Systems.

[86]  Dongpu Cao,et al.  Parallel driving in CPSS: a unified approach for transport automation and vehicle intelligence , 2017, IEEE/CAA Journal of Automatica Sinica.

[87]  Andreas Lawitzky,et al.  A Combined Model- and Learning-Based Framework for Interaction-Aware Maneuver Prediction , 2016, IEEE Transactions on Intelligent Transportation Systems.

[88]  Thomas A. Dingus,et al.  The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data , 2006 .

[89]  Makoto Itoh,et al.  Toward inference of driver's lane-change intent under cognitive distraction , 2010, Proceedings of SICE Annual Conference 2010.

[90]  Stefan Haufe,et al.  EEG potentials predict upcoming emergency brakings during simulated driving , 2011, Journal of neural engineering.

[91]  Mohan Trivedi,et al.  Investigating the relationships between gaze patterns, dynamic vehicle surround analysis, and driver intentions , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[92]  Michael Anthony Bauer,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 1 Portable and Scalable Vision-Based Vehicular Instrument , 2022 .

[93]  Takashi Imamura,et al.  Estimation for driver's intentions in straight road environment using Hidden Markov Models , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[94]  Christoph Stiller,et al.  Driver intent inference at urban intersections using the intelligent driver model , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[95]  M. Henning MODELLING DRIVER BEHAVIOUR IN ORDER TO INFER THE INTENTION TO CHANGE LANES , 2008 .

[96]  Xiaoli Zhang,et al.  Implicit Intention Communication in Human–Robot Interaction Through Visual Behavior Studies , 2017, IEEE Transactions on Human-Machine Systems.

[97]  E. Boer,et al.  A collaborative approach for human-centered driver assistance systems , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[98]  Mohan M. Trivedi,et al.  On the Roles of Eye Gaze and Head Dynamics in Predicting Driver's Intent to Change Lanes , 2009, IEEE Transactions on Intelligent Transportation Systems.

[99]  C. Heinze Modelling Intention Recognition for Intelligent Agent Systems , 2004 .

[100]  Guangdong Feng,et al.  Driving Intention Inference Based on Dynamic Bayesian Networks , 2014 .

[101]  Lie Lu,et al.  A generic framework of user attention model and its application in video summarization , 2005, IEEE Trans. Multim..

[102]  Chung Choo Chung,et al.  Robust Multirate Control Scheme With Predictive Virtual Lanes for Lane-Keeping System of Autonomous Highway Driving , 2015, IEEE Transactions on Vehicular Technology.

[103]  Karl Heinz Hoffmann,et al.  A mathematical model for predicting lane changes using the steering wheel angle. , 2014, Journal of safety research.

[104]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[105]  Mohan M. Trivedi,et al.  On Performance Evaluation of Driver Hand Detection Algorithms: Challenges, Dataset, and Metrics , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[106]  Christian Laugier,et al.  Evaluating risk at road intersections by detecting conflicting intentions , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[107]  Jan Kautz,et al.  Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[108]  Mohan M. Trivedi,et al.  Looking at Humans in the Age of Self-Driving and Highly Automated Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[109]  Stefan Haufe,et al.  Electrophysiology-based detection of emergency braking intention in real-world driving , 2014, Journal of neural engineering.

[110]  Anup Doshi,et al.  Lane change intent prediction for driver assistance: On-road design and evaluation , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[111]  Lee Skrypchuk,et al.  Characterization of Driver Neuromuscular Dynamics for Human–Automation Collaboration Design of Automated Vehicles , 2018, IEEE/ASME Transactions on Mechatronics.

[112]  Mohan M. Trivedi,et al.  Computer Vision and Image Understanding xxx (2011) xxx–xxx Contents lists available at SciVerse ScienceDirect Computer Vision and Image Understanding , 2022 .

[113]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[114]  Firas Lethaus,et al.  Using Pattern Recognition to Predict Driver Intent , 2011, ICANNGA.

[115]  Mohan M. Trivedi,et al.  On-road prediction of driver's intent with multimodal sensory cues , 2011, IEEE Pervasive Computing.

[116]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[117]  Ali Borji,et al.  What do eyes reveal about the mind?: Algorithmic inference of search targets from fixations , 2015, Neurocomputing.

[118]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[119]  Huiping Zhou,et al.  Eye Movement-Based Inference of Truck Driver's Intent of Changing Lanes , 2009 .

[120]  Mohan M. Trivedi,et al.  Continuous Head Movement Estimator for Driver Assistance: Issues, Algorithms, and On-Road Evaluations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[121]  Jian Huang,et al.  Control of Upper-Limb Power-Assist Exoskeleton Using a Human-Robot Interface Based on Motion Intention Recognition , 2015, IEEE Transactions on Automation Science and Engineering.

[122]  Ridha Soua,et al.  Situation awareness within the context of connected cars: A comprehensive review and recent trends , 2016, Inf. Fusion.

[123]  Mohan M. Trivedi,et al.  Tactical driver behavior prediction and intent inference: A review , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[124]  Klaus C. J. Dietmayer,et al.  Continuous Driver Intention Recognition with Hidden Markov Models , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[125]  Ridha Soua,et al.  Recent Trends in Driver Safety Monitoring Systems: State of the Art and Challenges , 2017, IEEE Transactions on Vehicular Technology.

[126]  Erik Steinmetz,et al.  Processing of Eye/Head-Tracking Data in Large-Scale Naturalistic Driving Data Sets , 2012, IEEE Transactions on Intelligent Transportation Systems.

[127]  Wei Zhan,et al.  Probabilistic Prediction of Vehicle Semantic Intention and Motion , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[128]  F. Steinkamp,et al.  Examining psychokinesis: the interaction of human intention with random number generators--a meta-analysis. , 2006, Psychological bulletin.

[129]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.