Eye movements predict driver reaction time to takeover request in automated driving: A real-vehicle study

Abstract For automated driving at SAE level 3 or lower, driver performance in responding to takeover requests (TORs) is decisive in providing system safety. A driver state monitoring system that can predict a driver’s performance in a TOR event will facilitate a safer control transition from vehicle to driver. This experimental study investigated whether driver eye-movement measured before a TOR can predict driving performance in a subsequent TOR event. We recruited participants (N = 36) to obtain realistic results in a real-vehicle study. In the experiment, drivers rode in an automated vehicle on a test track for about 32 min, and a critical TOR event occurred at the end of the drive. Eye movements were measured by a camera-based driver monitoring system, and five measures were extracted from the last 2-min epoch prior to the TOR event. The correlations between each eye-movement measure and driver reaction time were examined, and a multiple regression model was built using a stepwise procedure. The results showed that longer reaction time could be significantly predicted by a smaller number of large saccades, a greater number of medium saccades, and lower saccadic velocity. The implications of these relationships are consistent with previous studies. The present real-vehicle study can provide insights to the automotive industry in the search for a safer and more flexible interface between the automated vehicle and the driver.

[1]  X. Jessie Yang,et al.  Predicting Takeover Performance in Conditionally Automated Driving , 2020, CHI Extended Abstracts.

[2]  Motoyuki Akamatsu,et al.  Electrophysiological evaluation of attention in drivers and passengers: Toward an understanding of drivers’ attentional state in autonomous vehicles , 2016 .

[3]  Wolfgang Rosenstiel,et al.  Ready for Take-Over? A New Driver Assistance System for an Automated Classification of Driver Take-Over Readiness , 2017, IEEE Intelligent Transportation Systems Magazine.

[4]  T. Åkerstedt,et al.  Real driving at night – Predicting lane departures from physiological and subjective sleepiness , 2014, Biological Psychology.

[5]  Johan Engström,et al.  Sensitivity of eye-movement measures to in-vehicle task difficulty , 2005 .

[6]  Norbert K. Semmer,et al.  Taking the chance: Core self-evaluations predict relative gain in job resources following turnover , 2016, SpringerPlus.

[7]  Nadja Schömig,et al.  The Interaction Between Highly Automated Driving and the Development of Drowsiness , 2015 .

[8]  Wolfgang Rosenstiel,et al.  Ready for Take-Over? A New Driver Assistance System for an Automated Classification of Driver Take-Over Readiness , 2017 .

[9]  Motoyuki Akamatsu,et al.  The Relationship Between Drowsiness Level and Takeover Performance in Automated Driving , 2020, HCI.

[10]  Yong Gu Ji,et al.  How we can measure the non-driving-task engagement in automated driving: Comparing flow experience and workload. , 2018, Applied ergonomics.

[11]  Motoyuki Akamatsu,et al.  Effects of scheduled manual driving on drowsiness and response to take over request: A simulator study towards understanding drivers in automated driving. , 2019, Accident; analysis and prevention.

[12]  S. Martinez-Conde,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[13]  Kunihiro Hasegawa,et al.  Age-related differences in effects of non-driving related tasks on takeover performance in automated driving. , 2020, Journal of safety research.

[14]  T. Åkerstedt,et al.  Subjective sleepiness, simulated driving performance and blink duration: examining individual differences , 2006, Journal of sleep research.

[15]  Klaus Bengler,et al.  Take-over again: Investigating multimodal and directional TORs to get the driver back into the loop. , 2017, Applied ergonomics.

[16]  Francesco Bella,et al.  Driving simulator for speed research on two-lane rural roads. , 2008, Accident; analysis and prevention.

[17]  Riender Happee,et al.  Human factors of transitions in automated driving: A general framework and literature survey , 2016 .

[18]  David Crundall,et al.  Chapter 11 – Visual Attention While Driving: Measures of Eye Movements Used in Driving Research , 2011 .

[19]  R. R. Hocking,et al.  Selection of the Best Subset in Regression Analysis , 1967 .

[20]  Albert Kircher,et al.  Comparison of Two Eye-Gaze Based Real-Time Driver Distraction Detection Algorithms in a Small-Scale Field Operational Test , 2017 .

[21]  Yonggang Wang,et al.  How driving duration influences drivers’ visual behaviors and fatigue awareness: A naturalistic truck driving test Study , 2018 .

[22]  Yangdong Zhao,et al.  How eye movement and driving performance vary before, during, and after entering a long expressway tunnel: considering the differences of novice and experienced drivers under daytime and nighttime conditions , 2016, SpringerPlus.

[23]  Motoyuki Akamatsu,et al.  Effects of cognitive and visual loads on driving performance after take-over request (TOR) in automated driving. , 2020, Applied ergonomics.

[24]  Keiichi Uchimura,et al.  Driver Inattention Monitoring System for Intelligent Vehicles: A Review , 2009, IEEE Transactions on Intelligent Transportation Systems.

[25]  Marco Dozza,et al.  Modelling how drivers respond to a bicyclist crossing their path at an intersection: How do test track and driving simulator compare? , 2018, Accident; analysis and prevention.

[26]  Riender Happee,et al.  Modeling take-over performance in level 3 conditionally automated vehicles. , 2017, Accident; analysis and prevention.

[27]  P. Caffier,et al.  Experimental evaluation of eye-blink parameters as a drowsiness measure , 2003, European Journal of Applied Physiology.

[28]  K. Bengler,et al.  Vigilance Decrement and Passive Fatigue Caused by Monotony in Automated Driving , 2015 .

[29]  R. Happee,et al.  A human factors perspective on automated driving , 2017 .

[30]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[31]  Yuji Takeda,et al.  Assessing the Mental States of Fallback-Ready Drivers in Automated Driving by Electrooculography , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[32]  Leandro L. Di Stasi,et al.  Towards a driver fatigue test based on the saccadic main sequence: A partial validation by subjective report data , 2012 .

[33]  Klaus Bengler,et al.  Driver State Monitoring Systems-- Transferable Knowledge Manual Driving to HAD , 2015 .

[34]  P. Williams.  Processing Demands, Training, and the Vigilance Decrement , 1986 .

[35]  Julie M. Harris,et al.  A Link Between Attentional Function, Effective Eye Movements, and Driving Ability , 2016, Journal of experimental psychology. Human perception and performance.

[36]  Natasha Merat,et al.  Coming back into the loop: Drivers' perceptual-motor performance in critical events after automated driving. , 2017, Accident; analysis and prevention.

[37]  Andrew M Kwasniak,et al.  Driver distraction is more than just taking eyes off the road , 2011 .