Learning to use automation: Behavioral changes in interaction with automated driving systems

Abstract To evaluate human-machine interfaces for automated driving systems, a robust methodology is indispensable. The present driving simulator study investigated the effect of practice on behavioral measures (i.e., experimenter rating, reaction times, error rate) and the development of the preference-performance relationship for automated driving human-machine interfaces. In a within-subject design, N = 55 participants completed several transitions between manual, Level 2 and Level 3 automated driving. Behavioral measures followed the power law of practice with exception of transitions to manual and error rates for Level 3 automation. After the first block of interactions, preference no longer predicted performance. The preference-performance relationship remained stable after the second block of interactions, which is mainly due to a stabilization in behavioral parameters. To get a deeper insight into the evaluation of human-machine interfaces for automated driving, the results suggest the application of multi-method approaches. Furthermore, we found evidence for the influence of initial interactions for self-reported usability.

[1]  J. Krems,et al.  The evolution of mental model, trust and acceptance of adaptive cruise control in relation to initial information , 2013 .

[2]  Udo Konradt,et al.  Measuring Psychological Constructs Using Single-Item Scales: Answers to Experts' Comments and Additional Questions , 2013, Interact. Comput..

[3]  Jeff Sauro,et al.  The Factor Structure of the System Usability Scale , 2009, HCI.

[4]  Klaus Bengler,et al.  Rating of Take-Over Performance in Conditionally Automated Driving Using an Expert-Rating System , 2018 .

[5]  Alexandra Neukum,et al.  Controllability of partially automated driving functions - does it matter whether drivers are allowed to take their hands off the steering wheel? , 2015 .

[6]  Brad Mehlenbacher,et al.  On the Reading of Product Owner's Manuals: Perceptions and Product Complexity , 2002 .

[7]  Josef F. Krems,et al.  Keep Your Scanners Peeled , 2016, Hum. Factors.

[8]  Frederik Naujoks,et al.  How Usability Can Save the Day - Methodological Considerations for Making Automated Driving a Success Story , 2018, AutomotiveUI.

[9]  Frederik Naujoks,et al.  Towards guidelines and verification methods for automated vehicle HMIs , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[10]  Georg Jahn,et al.  Skill Acquisition While Operating In-Vehicle Information Systems: Interface Design Determines the Level of Safety-Relevant Distractions , 2009, Hum. Factors.

[11]  J. Krems,et al.  Learning and development of trust, acceptance and the mental model of ACC. A longitudinal on-road study , 2015 .

[12]  Katja Kircher,et al.  Vehicle-based studies of driving in the real world: the hard truth? , 2013, Accident; analysis and prevention.

[13]  Mark Vollrath,et al.  Practice makes better - Learning effects of driving with a multi-stage collision warning. , 2018, Accident; analysis and prevention.

[14]  Allen and Rosenbloom Paul S. Newell,et al.  Mechanisms of Skill Acquisition and the Law of Practice , 1993 .

[15]  William Payre,et al.  Impact of training and in-vehicle task performance on manual control recovery in an automated car , 2017 .

[16]  Olaf Köller,et al.  Umgang mit fehlenden Werten in der psychologischen Forschung : Probleme und Lösungen , 2007 .

[17]  J. Freeman,et al.  Comparison of self-report and objective measures of driving behavior and road safety: A systematic review. , 2018, Journal of safety research.

[18]  John R. Anderson,et al.  Knowledge Compilation: Mechanisms for the Automatization of Cognitive Skills. , 1980 .

[19]  Christopher D. Wickens,et al.  When Users Want What's not Best for Them , 1995 .

[20]  Jakob Nielsen,et al.  Measuring usability: preference vs. performance , 1994, CACM.

[21]  Manfred Amelang,et al.  Psychologische Diagnostik und Intervention , 2006 .

[22]  Natasha Merat,et al.  Were they in the loop during automated driving? Links between visual attention and crash potential , 2016, Injury Prevention.

[23]  Josef F. Krems,et al.  Prior Familiarization With Takeover Requests Affects Drivers’ Takeover Performance and Automation Trust , 2017, Hum. Factors.

[24]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[25]  Andreas Butz,et al.  A Bermuda Triangle?: A Review of Method Application and Triangulation in User Experience Evaluation , 2018, CHI.

[26]  Alexandra Neukum,et al.  Your Turn or My Turn?: Design of a Human-Machine Interface for Conditional Automation , 2016, AutomotiveUI.

[27]  Frederic M. Lord,et al.  On the Statistical Treatment of Football Numbers. , 1953 .

[28]  Gail M. Sullivan,et al.  Using Effect Size-or Why the P Value Is Not Enough. , 2012, Journal of graduate medical education.

[29]  John A. Michon,et al.  A critical view of driver behavior models: What do we know , 1985 .

[30]  Nadja Schömig,et al.  Expert-based controllability assessment of control transitions from automated to manual driving , 2018, MethodsX.

[31]  Mark R. Lepper,et al.  Perseverance in self-perception and social perception: biased attributional processes in the debriefing paradigm. , 1975, Journal of personality and social psychology.

[32]  Mark Vollrath,et al.  What Makes an Automated Vehicle a Good Driver? , 2018, CHI.

[33]  G. Norman.  Likert scales, levels of measurement and the “laws” of statistics , 2010, Advances in health sciences education : theory and practice.

[34]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[35]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[36]  E. Plan,et al.  Modeling and Simulation of Count Data , 2014, CPT: pharmacometrics & systems pharmacology.

[37]  Rosalee K. Meyer,et al.  Conceptual and Motor Learning in Music Performance , 2000, Psychological science.

[38]  A. Daffertshofer,et al.  Multiple time scales and multiform dynamics in learning to juggle. , 2004, Motor control.

[39]  David C. Howell,et al.  The Treatment of Missing Data , 2007 .

[40]  Markus Bühner.  Einführung in die Test- und Fragebogenkonstruktion , 2008 .

[41]  James R. Lewis,et al.  Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies , 2002, Int. J. Hum. Comput. Interact..

[42]  Frederik Naujoks,et al.  Use Cases for Assessing, Testing, and Validating the Human Machine Interface of Automated Driving Systems , 2018 .