Measuring Human-Automation Function Allocation

Function allocation is the design decision in which work functions are assigned to all agents in a team, both human and automated. Building on the preceding companion papers’ review of the requirements of effective function allocation and discussion of a computational framework for modeling function allocation, in this paper, we develop specific metrics of function allocation that can be derived from such models as well as from observations in high-fidelity human-in-the-loop simulations or real operations. These metrics span eight issues with function allocation: (a) workload, (b) stability of the work environment, (c) mismatches between responsibility and authority, (d) incoherency in function allocations, (e) interruptive automation, (f) automation’s boundary conditions, (g) function allocations limiting human adaptation to context, (h) and mission performance. Some of the metrics measure distinct issues whereas others assess different causes of issues that can manifest in similar ways; collectively, they are intended to be comprehensive in their ability to discriminate for a range of issues. Trade-offs may exist between these metrics, and they need to be examined collectively to identify potential trade-offs or conflicts between them. This paper continues the example given in the preceding companion paper, demonstrating how these metrics of function allocation can be assessed from computational simulations of an air transport flight deck through the descent phase of flight.

[1]  Ken Funk,et al.  Cockpit Task Management: Preliminary Definitions, Normative Theory, Error Taxonomy, and Design Recommendations , 1991 .

[2]  Kevin Taaffe,et al.  Human Performance Engineering , 2015 .

[3]  Renwick E. Curry,et al.  Flight-deck automation: promises and problems , 1980 .

[4]  Earl L. Wiener,et al.  Beyond the Sterile Cockpit , 1985 .

[5]  Raja Parasuraman,et al.  Monitoring an Automated System for a Single Failure: Vigilance and Task Complexity Effects , 1996, Hum. Factors.

[6]  Christopher D. Wickens,et al.  Mission Control of Multiple Unmanned Aerial Vehicles: A Workload Analysis , 2005, Hum. Factors.

[7]  Raja Parasuraman,et al.  Designing for Flexible Interaction Between Humans and Automation: Delegation Interfaces for Supervisory Control , 2007, Hum. Factors.

[8]  Amy R. Pritchett REVIEWING THE ROLE OF COCKPIT ALERTING SYSTEMS , 2001 .

[9]  Michael C. Dorneich,et al.  Towards a Characterization of Adaptive Systems: a Framework for Researchers and System Designers , 2017 .

[10]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[11]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  Ann M. Bisantz,et al.  Analysis of Cognitive Work , 2007 .

[13]  Karen M. Feigh,et al.  Incorporating multiple patterns of activity into the design of cognitive work support systems , 2011, Cognition, Technology & Work.

[14]  Barry H. Kantowitz,et al.  Attention and Mental Workload , 2000 .

[15]  Scott S Potter,et al.  Subjective Workload Assessment Technique (SWAT): A User's Guide , 1989 .

[16]  Karen M. Feigh,et al.  Modeling Work for Cognitive Work Support System Design in Operational Control Centers , 2007 .

[17]  Monika Richter,et al.  Cognition In The Wild , 2016 .

[18]  Amy R. Pritchett The System Safety Perspective , 2010 .

[19]  David Woods,et al.  1. How to make automated systems team players , 2002 .

[20]  Lisanne Bainbridge,et al.  Ironies of automation , 1982, Autom..

[21]  Charles E. Billings,et al.  Aviation Automation: The Search for A Human-centered Approach , 1996 .

[22]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[23]  Julie A. Jacko,et al.  Contextual Control Modes During an Airline Rescheduling Task , 2007 .

[24]  Sidney W. A. Dekker,et al.  MABA-MABA or Abracadabra? Progress on Human–Automation Co-ordination , 2002, Cognition, Technology & Work.

[25]  David D. Woods,et al.  Cognitive Technologies: The Design of Joint Human-Machine Cognitive Systems , 1986, AI Mag..

[26]  Richard W. Pew,et al.  Modeling Human and Organizational Behavior , 1998 .

[27]  Elliot E. Entin,et al.  Measures for Evaluation of Team Processes and Performance in Experiments and Exercises , 2022 .

[28]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[29]  Robert E Windom,et al.  Time... and time again. , 1986, Public health reports.

[30]  Erik Hollnagel,et al.  Barriers And Accident Prevention , 2004 .

[31]  Erik Hollnagel,et al.  Human Reliability Analysis: Context and Control , 1994 .

[32]  Karen M. Feigh,et al.  Design of cognitive work support systems for airline operations , 2007 .

[33]  A Kirlik Modeling Strategic Behavior in Human-Automation Interaction: Why an "Aid" Can (and Should) Go Unused , 1993, Human factors.

[34]  Daniel R. Ilgen,et al.  Multilevel theory of team decision making: Decision performance in teams incorporating distributed expertise. , 1995 .

[35]  Michael D. Harrison,et al.  Allocation of function: scenarios, context and the economics of effort , 2000, Int. J. Hum. Comput. Stud..

[36]  Nancy J. Cooke,et al.  Advances in Human Performance and Cognitive Engineering Research , 2002 .

[37]  N Moray,et al.  Trust, control strategies and allocation of function in human-machine systems. , 1992, Ergonomics.

[38]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[39]  Aline Chevalier,et al.  The use of Tholos software for combining measures of mental workload: Toward theoretical and methodological improvements , 2008, Behavior research methods.

[40]  D A Norman,et al.  The 'problem' with automation: inappropriate feedback and interaction, not 'over-automation'. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[41]  Peter A. Hancock,et al.  Allocating Functions Rationally between Humans and Machines , 1998 .