Effects of User Interfaces on Take-Over Performance: A Review of the Empirical Evidence

In automated driving, the user interface plays an essential role in guiding transitions between automated and manual driving. This literature review identified 25 studies that explicitly studied the effectiveness of user interfaces in automated driving. Our main selection criterion was how the user interface (UI) affected take-over performance in higher automation levels allowing drivers to take their eyes off the road (SAE3 and SAE4). We categorized user interface (UI) factors from an automated vehicle-related information perspective. Short take-over times are consistently associated with take-over requests (TORs) initiated by the auditory modality with high urgency levels. On the other hand, take-over requests directly displayed on non-driving-related task devices and augmented reality do not affect take-over time. Additional explanations of take-over situation, surrounding and vehicle information while driving, and take-over guiding information were found to improve situational awareness. Hence, we conclude that advanced user interfaces can enhance the safety and acceptance of automated driving. Most studies showed positive effects of advanced UI, but a number of studies showed no significant benefits, and a few studies showed negative effects of advanced UI, which may be associated with information overload. The occurrence of positive and negative results of similar UI concepts in different studies highlights the need for systematic UI testing across driving conditions and driver characteristics. Our findings propose future UI studies of automated vehicle focusing on trust calibration and enhancing situation awareness in various scenarios.

[1]  Lutz Lorenz,et al.  Designing take over scenarios for automated driving , 2014 .

[2]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[3]  Gerhard Rigoll,et al.  Exploring the Use of Augmented Reality Interfaces for Driver Assistance in Short-Notice Takeovers , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[4]  Mascha C. van der Voort,et al.  Supporting the Changing Driver’s Task: Exploration of Interface Designs for Supervision and Intervention in Automated Driving , 2016 .

[5]  J C F de Winter,et al.  Comparing spatially static and dynamic vibrotactile take-over requests in the driver seat. , 2017, Accident; analysis and prevention.

[6]  Fredrick Ekman,et al.  Creating appropriate trust for autonomous vehicles , 2015 .

[7]  Neville A. Stanton,et al.  Take-Over Time in Highly Automated Vehicles , 2018, Driver Reactions to Automated Vehicles.

[8]  Alexandra Neukum,et al.  Driver compliance to take-over requests with different auditory outputs in conditional automation. , 2017, Accident; analysis and prevention.

[9]  Alexandra Neukum,et al.  A Review of Non-driving-related Tasks Used in Studies on Automated Driving , 2017 .

[10]  Nadine B. Sarter,et al.  Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information , 2006, Hum. Factors.

[11]  Natasha Merat,et al.  Transition to manual: driver behaviour when resuming control from a highly automated vehicle , 2014 .

[12]  Klaus Bengler,et al.  An HMI Concept to Improve Driver's Visual Behavior and Situation Awareness in Automated Vehicle , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[13]  Sebastiaan M. Petermeijer,et al.  Driver response times to auditory, visual, and tactile take-over requests: A simulator study with 101 participants , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[14]  Milad Alemzadeh,et al.  Human-Computer Interaction: Overview on State of the Art , 2008 .

[15]  Boussaad Soualmi,et al.  Augmented reality versus classical HUD to take over from automated driving: An aid to smooth reactions and to anticipate maneuvers , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[16]  Philipp Wintersberger,et al.  Let Me Finish before I Take Over: Towards Attention Aware Device Integration in Highly Automated Vehicles , 2018, AutomotiveUI.

[17]  Yong Gu Ji,et al.  Non-driving-related tasks, workload, and takeover performance in highly automated driving contexts , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[18]  Alexandra Neukum,et al.  The effect of urgency of take-over requests during highly automated driving under distraction conditions , 2014 .

[19]  Göran Falkman,et al.  Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving , 2013, AutomotiveUI.

[20]  Joost C. F. de Winter,et al.  Determinants of take-over time from automated driving: A meta-analysis of 129 studies , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[21]  Frederik Diederichs,et al.  Take-Over Requests for Automated Driving , 2015 .

[22]  Helmut Krcmar,et al.  Improving take-over quality in automated driving by interrupting non-driving tasks , 2019, IUI.

[23]  Tal Oron-Gilad,et al.  Directional tactile alerts for take-over requests in highly-automated driving , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[24]  Wendy Ju,et al.  Behavioral Measurement of Trust in Automation , 2016 .

[25]  D A Norman,et al.  The 'problem' with automation: inappropriate feedback and interaction, not 'over-automation'. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[26]  Alexander Kunze,et al.  Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces , 2019, Ergonomics.

[27]  Susanne Boll,et al.  Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving , 2016, AutomotiveUI.

[28]  Klaus Bengler,et al.  “Take over!” How long does it take to get the driver back into the loop? , 2013 .

[29]  G. Burnett,et al.  Rebuilding drivers ’ situation awareness during take-over requests in level 3 automated cars , 2019 .

[30]  Johan Engström,et al.  Toward Computational Simulations of Behavior During Automated Driving Takeovers: A Review of the Empirical and Modeling Literatures , 2019, Hum. Factors.

[31]  Klaus Bengler,et al.  A method to improve driver's situation awareness in automated driving , 2017 .

[32]  Young Woo Kim,et al.  The effects of takeover request modalities on highly automated car control transitions. , 2019, Accident; analysis and prevention.

[33]  Tal Oron-Gilad,et al.  The Effects of Continuous Driving-Related Feedback on Drivers’ Response to Automation Failures , 2017 .

[34]  Manfred Tscheligi,et al.  Control Transition Interfaces in Semiautonomous Vehicles: A Categorization Framework and Literature Analysis , 2017, AutomotiveUI.

[35]  Claus Marberger,et al.  The Impact of Non-Driving Related Tasks on Take-over Performance in Conditionally Automated Driving – A Review of the Empirical Evidence , 2019 .

[36]  Klaus Bengler,et al.  Highly automated driving with a decoupled steering wheel , 2014 .

[37]  Wendy Ju,et al.  Timing of unstructured transitions of control in automated driving , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[38]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[39]  A. Matysiak,et al.  The impact of the interfaces of the driving automation system on a driver with regard to road traffic safety , 2018 .

[40]  Frank E. Pollick,et al.  Language-based multimodal displays for the handover of control in autonomous cars , 2015, AutomotiveUI.

[41]  Bobbie Seppelt,et al.  Potential Solutions to Human Factors Challenges in Road Vehicle Automation , 2016 .

[42]  Klaus Bengler,et al.  Taking Over Control From Highly Automated Vehicles in Complex Traffic Situations , 2016, Hum. Factors.

[43]  Annie Pauzié.  Head Up Display in Automotive: A New Reality for the Driver , 2015, HCI.

[44]  Stefan Brandenburg,et al.  Should the urgency of auditory-tactile takeover requests match the criticality of takeover situations? , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[45]  Stefan Brandenburg,et al.  Behavioral Changes to Repeated Takeovers in Highly Automated Driving: Effects of the Takeover-Request Design and the Nondriving-Related Task Modality , 2018, Hum. Factors.

[46]  Neville A. Stanton,et al.  Rolling Out the Red (and Green) Carpet: Supporting Driver Decision Making in Automation-to-Manual Transitions , 2019, IEEE Transactions on Human-Machine Systems.

[47]  Patricia R DeLucia,et al.  A Systematic Review and Meta-Analysis of Takeover Performance During Conditionally Automated Driving , 2020, Hum. Factors.

[48]  Omer Tsimhoni,et al.  Haptic seat for automated driving: preparing the driver to take control effectively , 2015, AutomotiveUI.

[49]  R. J. J. Jansen,et al.  Auditory Signal Design for Automatic Number Plate Recognition System , 2014, CHI 2014.