EyeCar: Modeling the Visual Attention Allocation of Drivers in Semi-Autonomous Vehicles

A safe transition between autonomous and manual control requires sustained visual attention of the driver for the perception and assessment of hazards in dynamic driving environments. Thus, drivers must retain a certain level of situation awareness to safely takeover. Understanding the visual attention allocation of drivers can pave the way for inferring their dynamic state of situational awareness. We propose a reinforcement and inverse-reinforcement learning framework for modeling passive drivers' visual attention allocation in semi-autonomous vehicles. The proposed approach measures the eye-movement of passive drivers to evaluate their responses to real-world rear-end collisions. The results show substantial individual differences in the eye fixation patterns by driving experience, even among fully attentive drivers. Experienced drivers were more attentive to the situational dynamics and were able to identify potentially hazardous objects before any collisions occurred. These models of visual attention could potentially be integrated into autonomous systems to continuously monitor and guide effective intervention. Keywords: Visual attention allocation; Situation awareness; Eye movements; Eye fixation; Eye-Tracking; Reinforcement Learning; Inverse Reinforcement Learning

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  G Underwood,et al.  Visual attention and the transition from novice to advanced driver , 2007, Ergonomics.

[3]  Konstantinos Makantasis,et al.  A Deep Reinforcement Learning Driving Policy for Autonomous Road Vehicles , 2019, ArXiv.

[4]  Wolfram Burgard,et al.  Learning driving styles for autonomous vehicles from demonstration , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Ridha Soua,et al.  Situation awareness within the context of connected cars: A comprehensive review and recent trends , 2016, Inf. Fusion.

[6]  Thomas A. Dingus,et al.  Naturalistic Driving Study: Technical Coordination and Quality Control , 2014 .

[7]  Cristian Sminchisescu,et al.  Reinforcement Learning for Visual Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  David Hsu,et al.  Exploration in Interactive Personalized Music Recommendation: A Reinforcement Learning Approach , 2013, TOMM.

[9]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[10]  Bobbie D. Seppelt,et al.  Keeping the driver in the loop: Dynamic feedback to support appropriate use of imperfect vehicle control automation , 2019, Int. J. Hum. Comput. Stud..

[11]  Lu Feng,et al.  The Effect of Whole-Body Haptic Feedback on Driver’s Perception in Negotiating a Curve , 2018, ArXiv.

[12]  D. Cremers,et al.  Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks , 2016, ArXiv.

[13]  Mary M Hayhoe,et al.  Task and context determine where you look. , 2016, Journal of vision.

[14]  Moritz Körber,et al.  Prediction of take-over time in highly automated driving by two psychometric tests , 2015 .

[15]  John K. Tsotsos,et al.  Towards the Quantitative Evaluation of Visual Attention Models Bottom−up Top-down Dynamic Static 0 0 0 , 2022 .

[16]  Dimitar Filev,et al.  Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning , 2019, Robotics Auton. Syst..

[17]  Andrea Palazzi,et al.  Predicting the Driver's Focus of Attention: The DR(eye)VE Project , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Victoria A Banks,et al.  Keep the driver in control: Automating automobiles of the future. , 2016, Applied ergonomics.

[19]  G. Rizzolatti,et al.  I Know What You Are Doing A Neurophysiological Study , 2001, Neuron.

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[22]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[23]  Wei Zhan,et al.  Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[24]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[25]  Er Meng Joo,et al.  A survey of inverse reinforcement learning techniques , 2012 .

[26]  Yuan Chang Leong,et al.  Dynamic Interaction between Reinforcement Learning and Attention in Multidimensional Environments , 2017, Neuron.

[27]  Natasha Merat,et al.  Preface to the Special Section on Human Factors and Automation in Vehicles , 2012, Hum. Factors.

[28]  Donald L. Fisher,et al.  Minimum Time to Situation Awareness in Scenarios Involving Transfer of Control from an Automated Driving Suite , 2016 .

[29]  Haobin Jiang,et al.  Learning Human-Like Trajectory Planning on Urban Two-Lane Curved Roads From Experienced Drivers , 2019, IEEE Access.

[30]  Sybil Derrible,et al.  Real-time accident detection: Coping with imbalanced data. , 2019, Accident; analysis and prevention.

[31]  John K. Tsotsos.  Toward a computational model of visual attention , 1995 .

[32]  Markus Wulfmeier,et al.  Maximum Entropy Deep Inverse Reinforcement Learning , 2015, 1507.04888.

[33]  Luxin Zhang,et al.  AGIL: Learning Attention from Human for Visuomotor Tasks , 2018, ECCV.

[34]  Tobias Vogelpohl,et al.  Asleep at the automated wheel-Sleepiness and fatigue during highly automated driving. , 2019, Accident; analysis and prevention.

[35]  Sadayuki Tsugawa,et al.  Automated Driving Systems: Common Ground of Automobiles and Robots , 2011, Int. J. Humanoid Robotics.

[36]  David Whitney,et al.  Predicting Driver Attention in Critical Situations , 2017, ACCV.

[37]  Stefan Roth,et al.  Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[38]  Verena Schmid,et al.  Solving the dynamic ambulance relocation and dispatching problem using approximate dynamic programming , 2012, Eur. J. Oper. Res..

[39]  Alan S. Minkoff.  A Markov Decision Model and Decomposition Heuristic for Dynamic Vehicle Dispatching , 1993, Oper. Res..

[40]  Byeongkeun Kang,et al.  A computational framework for driver's visual attention using a fully convolutional architecture , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[41]  Raja Parasuraman,et al.  Complacency and Bias in Human Use of Automation: An Attentional Integration , 2010, Hum. Factors.

[42]  Henrik I. Christensen,et al.  Computational visual attention systems and their cognitive foundations: A survey , 2010, TAP.

[43]  Michael L. Littman,et al.  Reinforcement learning improves behaviour from evaluative feedback , 2015, Nature.

[44]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[45]  John K. Tsotsos,et al.  Joint Attention in Driver-Pedestrian Interaction: from Theory to Practice , 2018, ArXiv.

[46]  Bobbie Seppelt,et al.  Potential Solutions to Human Factors Challenges in Road Vehicle Automation , 2016 .

[47]  Thomas B. Sheridan,et al.  A Closed-Loop Model of Operator Visual Attention, Situation Awareness, and Performance Across Automation Mode Transitions , 2017, Hum. Factors.

[48]  Andrea Palazzi,et al.  DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[49]  Nicholas J. Ward,et al.  OF VISION ENHANCEMENT SYSTEMS TO IMPROVE DRIVER SAFETY , 2016 .

[50]  J. Eggert,et al.  Managing the complexity of inner-city scenes: An efficient situation hypotheses selection scheme , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[51]  Bryan Reimer,et al.  Detection of brake lights while distracted: Separating peripheral vision from cognitive load , 2019, Attention, Perception, & Psychophysics.

[52]  A. Kiureghian,et al.  Aleatory or epistemic? Does it matter? , 2009 .

[53]  Etienne Perot,et al.  Deep Reinforcement Learning framework for Autonomous Driving , 2017, Autonomous Vehicles and Machines.

[54]  Martin Glavin,et al.  Vehicle Detection at Night Based on Tail-Light Detection , 2010 .

[55]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[56]  Thomas A. Dingus,et al.  The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data , 2006 .