Why Do I Have to Take Over Control? Evaluating Safe Handovers with Advance Notice and Explanations in HAD

In highly automated driving (HAD), it is still an open question how machines can safely hand over control to humans, and if an advance notice with additional explanations can be beneficial in critical situations. Conceptually, use of formal methods from AI – description logic (DL) and automated planning – in order to more reliably predict when a handover is necessary, and to increase the advance notice for handovers by planning ahead at runtime, can provide a technological support for explanations using natural language generation. However, in this work we address only the user’s perspective with two contributions: First, we evaluate our concept in a driving simulator study (N=23) and find that an advance notice and spoken explanations were preferred over classical handover methods. Second, we propose a framework and an example test scenario specific to handovers that is based on the results of our study.

[1]  Andreas Butz,et al.  Supporting Trust in Autonomous Driving , 2017 .

[2]  Paul Lukowicz,et al.  Automatic Calibration of Body Worn Acceleration Sensors , 2004, Pervasive.

[3]  Neville A. Stanton,et al.  Take-Over Time in Highly Automated Vehicles , 2018, Driver Reactions to Automated Vehicles.

[4]  Alfred Kobsa,et al.  User Models in Dialog Systems , 1989, Symbolic Computation.

[5]  J. P. Verma.  Repeated measures design for empirical researchers , 2015 .

[6]  Arne Jönsson,et al.  Perceived Usability and Cognitive Demand of Secondary Tasks in Spoken Versus Visual-Manual Automotive Interaction , 2016, INTERSPEECH.

[7]  Stefan Edelkamp,et al.  Automated Planning: Theory and Practice , 2007, Künstliche Intell..

[8]  Antonio Krüger,et al.  The Effects of Mobile Pedestrian Navigation Systems on the Concurrent Acquisition of Route and Survey Knowledge , 2004, Mobile HCI.

[9]  Vera Demberg,et al.  The Frequency of Rapid Pupil Dilations as a Measure of Linguistic Processing Difficulty , 2016, PloS one.

[10]  佐藤誠三郞,et al.  1997 , 1978, The Winning Cars of the Indianapolis 500.

[11]  Klaus Bengler,et al.  “Take over!” How long does it take to get the driver back into the loop? , 2013 .

[12]  G Pachiaudi,et al.  TRAFFIC AND TRANSPORT PSYCHOLOGY : THEORY AND APPLICATION. CHAPTER 18. SUBJECTIVE EVALUATION OF THE MENTAL WORKLOAD IN THE DRIVING CONTEXT , 1997 .

[13]  S. P. Marshall,et al.  The Index of Cognitive Activity: measuring cognitive workload , 2002, Proceedings of the IEEE 7th Conference on Human Factors and Power Plants.

[14]  윤재량.  2004 , 2019, The Winning Cars of the Indianapolis 500.

[15]  Antonio Krüger,et al.  Integrating Intra and Extra Gestures into a Mobile and Multimodal Shopping Assistant , 2005, Pervasive.

[16]  Catherine M. Burns,et al.  Autonomous Driving in the Real World: Experiences with Tesla Autopilot and Summon , 2016, AutomotiveUI.

[17]  Michael Weber,et al.  Autonomous driving: investigating the feasibility of car-driver handover assistance , 2015, AutomotiveUI.

[18]  Mustapha Mouloua,et al.  Measuring Trust of Autonomous Vehicles: A Development and Validation Study , 2015, HCI.

[19]  Susanne Boll,et al.  Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving , 2016, AutomotiveUI.

[20]  Wendy Ju,et al.  Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance , 2014, International Journal on Interactive Design and Manufacturing (IJIDeM).

[21]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[22]  Robert Dale,et al.  Building applied natural language generation systems , 1997, Natural Language Engineering.

[23]  Gottfried Zimmermann,et al.  Using gaze-based interactions in automated vehicles for increased road safety , 2019, AutomotiveUI.

[24]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[25]  Manfred Tscheligi,et al.  What We Can Learn from Pilots for Handovers and (De)Skilling in Semi-Autonomous Driving: An Interview Study , 2017, AutomotiveUI.

[26]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[27]  Philip R. Cohen,et al.  Tangible multimodal interfaces for safety-critical applications , 2004, CACM.

[28]  Claudia J. Stanny,et al.  Effects of distraction and experience on situation awareness and simulated driving , 2007 .

[29]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[30]  Marilyn Jager Adams,et al.  Situation Awareness and the Cognitive Management of Complex Systems , 1995, Hum. Factors.

[31]  Silvia Richter,et al.  The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks , 2010, J. Artif. Intell. Res..

[32]  Klaus Bengler,et al.  Ubernahmezeiten beim hochautomatisierten Autofahren , 2012 .

[33]  Thomas A. Dingus,et al.  The 100-Car Naturalistic Driving Study Phase II – Results of the 100-Car Field Experiment , 2006 .

[34]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[35]  Philipp Wintersberger,et al.  Attentive User Interfaces to Improve Multitasking and Take-Over Performance in Automated Driving: The Auto-Net of Things , 2019, Int. J. Mob. Hum. Comput. Interact..

[36]  Susanne Boll,et al.  From reading to driving: priming mobile users for take-over situations in highly automated driving , 2018, MobileHCI.

[37]  M. Endsley.  Automation and situation awareness. , 1996 .

[38]  Tina Mioch,et al.  Driver Readiness Model for Regulating the Transfer from Automation to Human Control , 2017, IUI.

[39]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[40]  John D. Lee,et al.  Trust in Automation: Designing for Appropriate Reliance , 2004, Hum. Factors.

[41]  Moritz Körber,et al.  Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation , 2018, Advances in Intelligent Systems and Computing.

[42]  Shamsi T. Iqbal,et al.  Priming Drivers before Handover in Semi-Autonomous Cars , 2017, CHI.

[43]  Natasha Merat,et al.  Transition to manual: driver behaviour when resuming control from a highly automated vehicle , 2014 .

[44]  Wendy Ju,et al.  Emergency, Automation Off: Unstructured Transition Timing for Distracted Drivers of Automated Vehicles , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[45]  Antonio Krüger,et al.  ReflectiveSigns: Digital Signs That Adapt to Audience Attention , 2009, Pervasive.

[46]  Joost C. F. de Winter,et al.  Determinants of take-over time from automated driving: A meta-analysis of 129 studies , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[47]  Yuan Shen,et al.  To Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehicles , 2020, ArXiv.

[48]  E. Williams.  Experimental Designs Balanced for the Estimation of Residual Effects of Treatments , 1949 .

[49]  X. Jessie Yang,et al.  From Manual Driving to Automated Driving: A Review of 10 Years of AutoUI , 2019, AutomotiveUI.

[50]  Shamsi T. Iqbal,et al.  Interrupted by my car? Implications of interruption and interleaving research for automated vehicles , 2019, Int. J. Hum. Comput. Stud..

[51]  Mark Colley,et al.  Don't you see them?: towards gaze-based interaction adaptation for driver-vehicle cooperation , 2019, AutomotiveUI.

[52]  Albrecht Schmidt,et al.  Making use of drivers' glances onto the screen for explicit gaze-based interaction , 2010, AutomotiveUI.

[53]  Antonio Krüger,et al.  A User Modeling Markup Language (UserML) for Ubiquitous Computing , 2003, User Modeling.

[54]  Øyvind Smogeli,et al.  Trustworthy versus Explainable AI in Autonomous Vessels , 2020, Proceedings of the International Seminar on Safety and Security of Autonomous Vessels (ISSAV) and European STAMP Workshop and Conference (ESWC) 2019.

[55]  Jeremy D Sudweeks,et al.  An Analysis of Driver Inattention Using a Case-Crossover Approach On 100-Car Data: Final Report , 2010 .

[56]  Christian A. Müller,et al.  Measuring linguistically-induced cognitive load during driving using the ConTRe task , 2013, AutomotiveUI.

[57]  Ian Horrocks,et al.  An Introduction to Description Logic , 2017 .

[58]  Florence March,et al.  2016 , 2016, Affair of the Heart.

[59]  Tsvi Kuflik,et al.  Adaptive, intelligent presentation of information for the museum visitor in PEACH , 2007, User Modeling and User-Adapted Interaction.

[60]  Yvonne Rogers,et al.  How to nudge in Situ: designing lambent devices to deliver salient information in supermarkets , 2011, UbiComp '11.

[61]  Jacques M. B. Terken,et al.  Compatibility between Trust and Non-Driving Related Tasks in UI Design for Highly and Fully Automated Driving , 2016, AutomotiveUI.

[62]  Charles A. Green,et al.  Human Factors Issues Associated with Limited Ability Autonomous Driving Systems: Drivers’ Allocation of Visual Attention to the Forward Roadway , 2017 .

[63]  Vera Demberg,et al.  Language and cognitive load in a dual task environment , 2013, CogSci.