Exploring the benefits of conversing with a digital voice assistant during automated driving: A parametric duration model of takeover time

Abstract Vehicle automation allows drivers to disengage from driving causing a potential decline in their alertness. One of the major challenges of highly automated vehicles is to ensure a timely (with respect to safety and situation awareness) takeover in such conditions. For this purpose, the current study investigated the role of an in-vehicle digital voice-assistant (VA) in conditionally automated vehicles, offering spoken discourse relating specifically to contextual factors, such as the traffic situation and road environment. The study involved twenty-four participants, each taking two drives (counterbalanced): with VA and without VA, in a driving simulator. Participants were required to takeover vehicle control following the issuance of a takeover request (TOR) near the end of each drive. A parametric duration model was adopted to find the key factors determining takeover time (TOT). Paired comparisons showed higher alertness and higher active workload (mean NASA-TLX rating) during automation when accompanied by the VA. Paired t-test comparison of gaze behavior prior to takeover showed significantly higher instances of checking traffic signal, roadside objects, and the roadway during the drive with VA, indicating higher situation awareness. The parametric model indicated that the VA increased the likelihood of making a timely takeover by 39%. There was also some evidence suggesting that male drivers are likely to resume control 1.21 times earlier than female drivers. The study findings highlight the benefits of adopting a digital voice assistant to keep the drivers alert and aware about the recent traffic environment in partially automated vehicles.

[1]  Changxu Wu,et al.  The Effects of Lead Time of Take-Over Request and Nondriving Tasks on Taking-Over Control of Automated Vehicles , 2018, IEEE Transactions on Human-Machine Systems.

[2]  Brian W. Sloboda,et al.  Statistical and Econometric Methods for Transportation Data Analysis , 2012 .

[3]  Samuel G. Charlton,et al.  Drivers’ response to speed warnings provided by a smart phone app , 2020 .

[4]  Gerald Matthews,et al.  The relationship between cell phone use and management of driver fatigue: It's complicated. , 2017, Journal of safety research.

[5]  Carryl L. Baldwin,et al.  Driver fatigue: The importance of identifying causal factors of fatigue when considering detection and countermeasure technologies , 2009 .

[6]  S. Washington,et al.  Statistical and Econometric Methods for Transportation Data Analysis , 2010 .

[7]  Mercedes Castro-Nuño,et al.  Exploring the relationship between truck load capacity and traffic accidents in the European Union , 2016 .

[8]  Yvonne Tran,et al.  The influence of fatigue on brain activity , 2012 .

[9]  Leslie E. Papke,et al.  Econometric Methods for Fractional Response Variables with an Application to 401(K) Plan Participation Rates , 1993 .

[10]  Klaus Bengler,et al.  Drowsiness in Conditional Automation: Proneness, diagnosis and driving performance effects , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[11]  P. Gomez,et al.  Eye gaze behavior during affective picture viewing: Effects of motivational significance, gender, age, and repeated exposure , 2019, Biological Psychology.

[12]  Nagendra R. Velaga,et al.  Modelling the relationship between different Blood Alcohol Concentrations and reaction time of young and mature drivers , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[13]  Xuesong Wang,et al.  Driver drowsiness detection based on non-intrusive metrics considering individual specifics. , 2016, Accident; analysis and prevention.

[14]  Lee Skrypchuk,et al.  Steering the conversation: A linguistic exploration of natural language interactions with a digital assistant during simulated driving. , 2017, Applied ergonomics.

[15]  Lisa J. Molnar,et al.  Understanding trust and acceptance of automated vehicles: An exploratory simulator study of transfer of control between automated and manual driving , 2018, Transportation Research Part F: Traffic Psychology and Behaviour.

[16]  Alice Gross,et al.  Driver glance behavior towards displayed images on in-vehicle information systems under real driving conditions , 2020 .

[17]  Marcin Kosobudzki,et al.  Application of eye-tracking in the testing of drivers: A review of research. , 2015, International journal of occupational medicine and environmental health.

[18]  R. Young Drowsy driving increases severity of safety-critical events and is decreased by cell phone conversation , 2013 .

[19]  Cassie J. Hilditch,et al.  Sleep inertia associated with a 10-min nap before the commute home following a night shift: A laboratory simulation study. , 2017, Accident; analysis and prevention.

[20]  Mark Wetton,et al.  Hazard perception in novice and experienced drivers: the effects of sleepiness. , 2009, Accident; analysis and prevention.

[21]  Donald L. Fisher,et al.  Experienced drivers are quicker to achieve situation awareness than inexperienced drivers in situations of transfer of control within a Level 3 autonomous environment , 2016 .

[22]  Johan Engström,et al.  Toward Computational Simulations of Behavior During Automated Driving Takeovers: A Review of the Empirical and Modeling Literatures , 2019, Hum. Factors.

[23]  Feng Guo,et al.  Keep your eyes on the road: young driver crash risk increases according to duration of distraction. , 2014, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[24]  P. Hanly,et al.  Measurement properties of the Epworth sleepiness scale. , 2003, Sleep medicine.

[25]  Klaus Bengler,et al.  “Take over!” How long does it take to get the driver back into the loop? , 2013 .

[26]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[27]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[28]  Chaozhong Wu,et al.  The effect of fatigue driving on car following behavior , 2016 .

[29]  Andy P. Field,et al.  Discovering Statistics Using SPSS , 2000 .

[30]  Edward M. Hitchcock,et al.  Active and passive fatigue in simulated driving: discriminating styles of workload regulation and their safety impacts. , 2013, Journal of experimental psychology. Applied.

[31]  Miguel A. Perez,et al.  Evaluating driver eye glance behavior and secondary task engagement while using driving automation systems. , 2020, Accident; analysis and prevention.

[32]  David R. Large,et al.  Driven to discussion: engaging drivers in conversation with a digital assistant as a countermeasure to passive task-related fatigue , 2018 .

[33]  David R. Large,et al.  A Longitudinal Simulator Study to Explore Drivers' Behaviour in Level 3 Automated Vehicles , 2019, AutomotiveUI.

[34]  Simon Washington,et al.  The impact of mobile phone distraction on the braking behaviour of young drivers : a hazard-based duration model , 2014 .

[35]  D. Strayer,et al.  Passenger and Cell-Phone Conversations in Simulated Driving , 2004, Journal of experimental psychology. Applied.

[36]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[37]  Michael G. Lenné,et al.  The relative importance of real-time in-cab and external feedback in managing fatigue in real-world commercial transport operations , 2017, Traffic injury prevention.

[38]  G. Burnett,et al.  Exploring the effectiveness of a digital voice assistant to maintain driver alertness in partially automated vehicles , 2021, Traffic injury prevention.

[39]  Tobias Vogelpohl,et al.  Asleep at the automated wheel-Sleepiness and fatigue during highly automated driving. , 2019, Accident; analysis and prevention.

[40]  Natasha Merat,et al.  Behavioural changes in drivers experiencing highly-automated vehicle control in varying traffic conditions , 2013 .

[41]  Michael K. Scullin,et al.  Sleep, Cognition, and Normal Aging , 2015, Perspectives on psychological science : a journal of the Association for Psychological Science.

[42]  Kathrin Zeeb,et al.  Is take-over time all that matters? The impact of visual-cognitive load on driver take-over quality after conditionally automated driving. , 2016, Accident; analysis and prevention.

[43]  Marco Dozza,et al.  What factors influence drivers' response time for evasive maneuvers in real traffic? , 2013, Accident; analysis and prevention.

[44]  Motoyuki Akamatsu,et al.  Effects of scheduled manual driving on drowsiness and response to take over request: A simulator study towards understanding drivers in automated driving. , 2019, Accident; analysis and prevention.

[45]  David R. Large,et al.  Assessing Cognitive Demand during Natural Language Interactions with a Digital Driving Assistant , 2016, AutomotiveUI.

[46]  D. Shinar,et al.  Effects of uncertainty, transmission type, driver age and gender on brake reaction and movement time. , 2002, Journal of safety research.

[47]  Dario Babić,et al.  Driver eye movements in relation to unfamiliar traffic signs: An eye tracking study. , 2020, Applied ergonomics.

[48]  Cassie J. Hilditch,et al.  Do night naps impact driving performance and daytime recovery sleep? , 2017, Accident; analysis and prevention.

[49]  Catherine Neubauer,et al.  Fatigue and Voluntary Utilization of Automation in Simulated Driving , 2012, Hum. Factors.

[50]  Rebecca A. Grier,et al.  Fundamental dimensions of subjective state in performance settings: task engagement, distress, and worry. , 2002, Emotion.

[51]  Pushpa Choudhary,et al.  Analysis of vehicle-based lateral performance measures during distracted driving due to phone use , 2017 .

[52]  W. Dement,et al.  Quantification of sleepiness: a new approach. , 1973, Psychophysiology.

[53]  M. Ohayon,et al.  Demographic factors, fatigue, and driving accidents: An examination of the published literature. , 2011, Accident; analysis and prevention.

[54]  Catherine Neubauer,et al.  Fatigue in the Automated Vehicle , 2014 .

[55]  Mark S. Young,et al.  Malleable Attentional Resources Theory: A New Explanation for the Effects of Mental Underload on Performance , 2002, Hum. Factors.

[56]  K. Bengler,et al.  Vigilance Decrement and Passive Fatigue Caused by Monotony in Automated Driving , 2015 .

[57]  Natasha Merat,et al.  Transition to manual: driver behaviour when resuming control from a highly automated vehicle , 2014 .

[58]  Joost C. F. de Winter,et al.  Determinants of take-over time from automated driving: A meta-analysis of 129 studies , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[59]  G. Matthews Towards a transactional ergonomics for driver stress and fatigue , 2002 .

[60]  Natasha Merat,et al.  Coming back into the loop: Drivers' perceptual-motor performance in critical events after automated driving. , 2017, Accident; analysis and prevention.

[61]  T. Åkerstedt,et al.  Work and rest sleep schedules of 227 European truck drivers. , 2002, Sleep medicine.

[62]  Donald L. Fisher,et al.  Minimum Time to Situation Awareness in Scenarios Involving Transfer of Control from an Automated Driving Suite , 2016 .

[63]  Simon Washington,et al.  A parametric duration model of the reaction times of drivers distracted by mobile phone conversations. , 2014, Accident; analysis and prevention.

[64]  Gerald Matthews,et al.  Dangerous intersections? A review of studies of fatigue and distraction in the automated vehicle. , 2019, Accident; analysis and prevention.

[65]  A. Muzet,et al.  Effect of driving duration and partial sleep deprivation on subsequent alertness and performance of car drivers , 2005, Physiology & Behavior.

[66]  R. González-Reyes,et al.  Sleep evaluation scales and questionaries: a review. , 2008, Actas espanolas de psiquiatria.

[67]  Kathrin Zeeb,et al.  What determines the take-over time? An integrated model approach of driver take-over after automated driving. , 2015, Accident; analysis and prevention.

[68]  Zhenji Lu,et al.  How much time do drivers need to obtain situation awareness? A laboratory-based study of automated driving. , 2017, Applied ergonomics.

[69]  A. Muzet,et al.  Oculomotor changes due to road events during prolonged monotonous simulated driving , 2005, Biological Psychology.

[70]  D Shinar,et al.  Self-reports of safe driving behaviors in relationship to sex, age, education and income in the US adult driving population. , 2001, Accident; analysis and prevention.

[71]  Willem Vlakveld,et al.  Situation awareness increases when drivers have more time to take over the wheel in a Level 3 automated car: A simulator study , 2018, Transportation Research Part F: Traffic Psychology and Behaviour.

[72]  Gerald Matthews,et al.  The Handbook of Operator Fatigue , 2012 .

[73]  Thomas A. Dingus,et al.  The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data , 2006 .

[74]  Riender Happee,et al.  Modeling take-over performance in level 3 conditionally automated vehicles. , 2017, Accident; analysis and prevention.

[75]  Paul C. Schutte,et al.  Human automation interaction , 2018 .

[76]  S. Puttonen,et al.  Sleepiness, sleep, and use of sleepiness countermeasures in shift-working long-haul truck drivers. , 2015, Accident; analysis and prevention.

[77]  Said M. Easa,et al.  Modeling Reaction Time in Car-Following Behaviour Based on Human Factors , 2009 .

[78]  A. Craig,et al.  A critical review of the psychophysiology of driver fatigue , 2001, Biological Psychology.

[79]  Paul Atchley,et al.  A Strategically Timed Verbal Task Improves Performance and Neurophysiological Alertness During Fatiguing Drives , 2014, Hum. Factors.

[80]  Nagendra R. Velaga,et al.  Effects of Partial Sleep Deprivation on Braking Response of Drivers in Hazard Scenarios. , 2020, Accident; analysis and prevention.

[81]  T. Åkerstedt,et al.  Subjective and objective sleepiness in the active individual. , 1990, The International journal of neuroscience.

[82]  S Lings,et al.  Assessing driving capability: a method for individual testing: the significance of paraparesis inferior studied in a controlled experiment. , 1991, Applied ergonomics.

[83]  Alicja Bortkiewicz,et al.  Application of eye-tracking in the testing of drivers: A review of research. , 2015, International journal of occupational medicine and environmental health.

[84]  Pushpa Choudhary,et al.  Mobile phone use during driving: Effects on speed and effectiveness of driver compensatory behaviour. , 2017, Accident; analysis and prevention.

[85]  Marieke Hendrikje Martens,et al.  Do Familiarity and Expectations Change Perception? Drivers' Glances and Response to Changes , 2007 .

[86]  Cylie M. Williams,et al.  A novel counterbalanced implementation study design: methodological description and application to implementation research , 2019, Implementation Science.

[87]  Dick de Waard,et al.  Behavioural impacts of Advanced Driver Assistance Systems–an overview , 2019, European Journal of Transport and Infrastructure Research.

[88]  Pushpa Choudhary,et al.  Modelling driver distraction effects due to mobile phone use on reaction time , 2017 .