From Here to Autonomy

As autonomous and semiautonomous systems are developed for automotive, aviation, cyber, robotics and other applications, the ability of human operators to effectively oversee and interact with them when needed poses a significant challenge. An automation conundrum exists in which as more autonomy is added to a system, and its reliability and robustness increase, the lower the situation awareness of human operators and the less likely that they will be able to take over manual control when needed. The human–autonomy systems oversight model integrates several decades of relevant autonomy research on operator situation awareness, out-of-the-loop performance problems, monitoring, and trust, which are all major challenges underlying the automation conundrum. Key design interventions for improving human performance in interacting with autonomous systems are integrated in the model, including human–automation interface features and central automation interaction paradigms comprising levels of automation, adaptive automation, and granularity of control approaches. Recommendations for the design of human–autonomy interfaces are presented and directions for future research discussed.

[1]  Jeffrey D. Karpicke,et al.  The Power of Testing Memory Basic Research and Implications for Educational Practice , 2006 .

[2]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[3]  Hiroshi Furukawa,et al.  Supporting System-Centered View of Operators Through Ecological Interface Design: Two Experiments on Human-Centered Automation , 2003 .

[4]  M. Endsley Automation and situation awareness. , 1996 .

[5]  Mica R. Endsley Situation awareness: operationally necessary and scientifically grounded , 2015, Cognition, Technology & Work.

[6]  Raja Parasuraman,et al.  Display Integration Enhances Information Sampling and Decision Making in Automated Fault Management in a Simulated Spaceflight Micro-World , 2002, Aviation, space, and environmental medicine.

[7]  N. Moray,et al.  Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. , 1996, Ergonomics.

[8]  Mark W. Scerbo,et al.  Theoretical Perspectives on Adaptive Automation , 2019, Human Performance in Automated and Autonomous Systems.

[9]  Charles E. Billings,et al.  Human-centered aircraft automation: A concept and guidelines , 1991 .

[10]  Christopher D. Wickens,et al.  Target Cuing in Visual Search: The Effects of Conformality and Display Location on the Allocation of Visual Attention , 1999, Hum. Factors.

[11]  James L. Szalma,et al.  A Meta-Analysis of Factors Influencing the Development of Trust in Automation , 2016, Hum. Factors.

[12]  Amy R. Pritchett Pilot non-conformance to alerting system commands , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[13]  Mustapha Mouloua,et al.  Automation and Human Performance : Theory and Applications , 1996 .

[14]  Douglas A. Wiegmann,et al.  Cognitive Anchoring on Self-Generated Decisions Reduces Operator Reliance on Automated Diagnostic Aids , 2005, Hum. Factors.

[15]  Mica R. Endsley,et al.  The Effect of Task Load and Shared Displays on Team Situation Awareness , 2000 .

[16]  F. Gobet,et al.  The Cambridge handbook of expertise and expert performance , 2006 .

[17]  P. Hancock,et al.  Human Mental Workload , 1988 .

[18]  Thomas B. Sheridan,et al.  Human and Computer Control of Undersea Teleoperators , 1978 .

[19]  Neville Moray,et al.  Monitoring behavior and supervisory control , 1986 .

[20]  Michelle M. Robertson,et al.  Situation awareness in aircraft maintenance teams , 2000 .

[21]  L. Young On adaptive manual control. , 1969, Ergonomics.

[22]  Nadine B. Sarter,et al.  Supporting Informed Consent in Human-Machine Collaboration: The Role of Conflict Type, Time Pressure, and Display Design , 1999 .

[23]  Christopher D. Wickens,et al.  The benefits of imperfect diagnostic automation: a synthesis of the literature , 2007 .

[24]  P A Hancock,et al.  Imposing limits on autonomous systems , 2017, Ergonomics.

[25]  Mark W. Scerbo,et al.  A Brain-Based Adaptive Automation System and Situation Awareness: The Role of Complacency Potential , 2003 .

[26]  M. Endsley,et al.  Individual Differences in Pilot Situation Awareness , 1994 .

[27]  Glenn F. Wilson,et al.  Performance Enhancement in an Uninhabited Air Vehicle Task Using Psychophysiologically Determined Adaptive Aiding , 2007, Hum. Factors.

[28]  Jeffrey M. Bradshaw,et al.  Ten Challenges for Making Automation a "Team Player" in Joint Human-Agent Activity , 2004, IEEE Intell. Syst..

[29]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[30]  Eric Davey Automation and Human Performance: Theory and Applications by Raja Parasuraman & Mustapha Mouloua 1996, 497 pages, $89.95 Mahwah, NJ: Lawrence Erlbaum Associates ISBN 0-8058-1616-X , 1998 .

[31]  B. Rouse William,et al.  Adaptive Aiding for Human/Computer Control , 1988 .

[32]  David B. Kaber,et al.  Situation Awareness in Driving While Using Adaptive Cruise Control and a Cell Phone , 2005 .

[33]  David B. Kaber,et al.  Adaptive Automation of a Dynamic Control Task Based on Secondary Task Workload Measurement , 1999 .

[34]  Robert P. Goldman,et al.  "Tasking" Interfaces for Flexible Interaction with Automa- tion: Keeping the Operator in Control , 1999 .

[35]  Linda Onnasch,et al.  Human Performance Consequences of Automated Decision Aids , 2012 .

[36]  Endsley,et al.  A model of inter- and intrateam situation awareness: implications for design, training and measurement , 2001 .

[37]  David B. Kaber,et al.  Design of Automation for Telerobots and the Effect on Performance, Operator Situation Awareness, and Subjective Workload , 2000 .

[38]  Philip J. Smith,et al.  Design of a Cooperative Problem-Solving System for En-Route Flight Planning: An Empirical Evaluation , 1994 .

[39]  John M. Reising,et al.  The Human-Electronic Crew: Can We Trust The Team? , 1995 .

[40]  Huiyang Li,et al.  Using Modeling and Simulation to Predict Operator Performance and Automation-Induced Complacency With Robotic Automation , 2015, Hum. Factors.

[41]  Stephanie Guerlain,et al.  Interactive Critiquing as a Form of Decision Support: An Empirical Evaluation , 1999, Hum. Factors.

[42]  David Woods,et al.  Incidents – Markers of Resilience or Brittleness? , 2017 .

[43]  N. J. Slamecka,et al.  The Generation Effect: Delineation of a Phenomenon , 1978 .

[44]  Stephen M Fiore,et al.  Augmenting team cognition in human-automation teams performing in complex operational environments. , 2007, Aviation, space, and environmental medicine.

[45]  Raja Parasuraman,et al.  Effects of Imperfect Automation on Decision Making in a Simulated Command and Control Task , 2007, Hum. Factors.

[46]  John D. Lee,et al.  Cooperation in Human-Agent Systems to Support Resilience , 2016, Hum. Factors.

[47]  Dietrich Manzey,et al.  Human Performance Consequences of Automated Decision Aids in States of Sleep Loss , 2011, Hum. Factors.

[48]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[49]  Toshiyuki Inagaki,et al.  Attention and complacency , 2000 .

[50]  Arathi Sethumadhavan Effects of Automation Types on Air Traffic Controller Situation Awareness and Performance , 2009 .

[51]  Natasha Merat,et al.  Control Task Substitution in Semiautomated Driving , 2012, Hum. Factors.

[52]  U. Krogmann From Automation to Autonomy-Trends Towards Autonomous Combat Systems , 2000 .

[53]  Kenneth R. Allendoerfer,et al.  Effect of Free Flight Conditions on Controller Performance, Workload, and Situation Awareness , 1997 .

[54]  David Woods,et al.  The Risks of Autonomy , 2016 .

[55]  Eduardo Salas,et al.  Situation Awareness in Team Performance: Implications for Measurement and Training , 1995, Hum. Factors.

[56]  Mica R. Endsley,et al.  Designing for Situation Awareness : An Approach to User-Centered Design , 2003 .

[57]  Abuse Humans and Automation : Use , Misuse , Disuse , , 2008 .

[58]  Nancy J. Cooke,et al.  Measuring Team Knowledge: A Window to the Cognitive Underpinnings of Team Performance , 2003 .

[59]  Earl S. Stein,et al.  The Use of Predictive Displays for Aiding Controller Situation Awareness , 1999 .

[60]  Mica R. Endsley,et al.  Information Presentation for Expert Systems in Future Fighters Aircraft , 1994 .

[61]  Céline McKeown,et al.  Designing for Situation Awareness: An Approach to User-Centered Design , 2013 .

[62]  Huiyang Li,et al.  Stages and Levels of Automation in Support of Space Teleoperations , 2014, Hum. Factors.

[63]  Bruce G. Coury,et al.  Status or Recommendation: Selecting the Type of Information for Decision Aiding , 1990 .

[64]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[65]  Nadine B. Sarter,et al.  Good Vibrations: Tactile Feedback in Support of Attention Allocation and Human-Automation Coordination in Event-Driven Domains , 1999, Hum. Factors.

[66]  Mica R. Endsley,et al.  Providing understanding of the behavior of feedforward neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[67]  Lisanne Bainbridge,et al.  Ironies of automation , 1982, Autom..

[68]  Charles E. Billings,et al.  Aviation Automation: The Search for A Human-centered Approach , 1996 .

[69]  Renwick E. Curry,et al.  Flight-deck automation: promises and problems , 1980 .

[70]  Jessie Y. C. Chen,et al.  A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction , 2011, Hum. Factors.

[71]  Raja Parasuraman,et al.  The Role of the Air Traffic Controller in Future Air Traffic Management: An Empirical Study of Active Control versus Passive Monitoring , 2001, Hum. Factors.

[72]  Christopher D. Wickens,et al.  The Effects of Participatory Mode and Task Workload on the Detection of Dynamic System Failures , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[73]  Deborah Lee,et al.  Are Well-Calibrated Users Effective Users? Associations Between Calibration of Trust and Performance on an Automation-Aided Task , 2015, Hum. Factors.

[74]  Mark W. Scerbo,et al.  Effects of a Psychophysiological System for Adaptive Automation on Performance, Workload, and the Event-Related Potential P300 Component , 2003, Hum. Factors.

[75]  Mica R. Endsley,et al.  Distribution of Attention, Situation Awareness and Workload in a Passive Air Traffic Control Task: Implications for Operational Errors and Automation , 1998 .

[76]  Raja Parasuraman,et al.  Performance Consequences of Automation-Induced 'Complacency' , 1993 .

[77]  David B. Kaber,et al.  The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task , 2004 .

[78]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[79]  Mica R. Endsley,et al.  Expertise and Situation Awareness , 2006 .

[80]  John D. Lee,et al.  Trust in Automation: Designing for Appropriate Reliance , 2004, Hum. Factors.

[81]  Nadine B. Sarter,et al.  Supporting Decision Making and Action Selection under Time Pressure and Uncertainty: The Case of In-Flight Icing , 2001, Hum. Factors.

[82]  Mica R. Endsley,et al.  Towards a New Paradigm for Automation: Designing for Situation Awareness , 1995 .

[83]  P. Hancock,et al.  In search of vigilance: the problem of iatrogenically created psychological phenomena. , 2013, The American psychologist.

[84]  Raja Parasuraman,et al.  Automation in Future Air Traffic Management: Effects of Decision Aid Reliability on Controller Performance and Mental Workload , 2005, Hum. Factors.

[85]  Huiyang Li,et al.  Human Performance Consequences of Stages and Levels of Automation , 2014, Hum. Factors.