Literature-Research of Design Guidance for Intelligent Adaptive Automation and Interfaces

Abstract : Human-machine system performance can be significantly improved by using technologies that can intelligently adapt the operator machine interface (OMI) and/or task automation provided to the operator in accordance with both the external context (i.e., task environment) and internal context (i.e., operator state). However, a lack of established design guidelines presents a significant challenge to the effective design of Intelligent Adaptive Systems (IASs). An extensive literature review was conducted to examine existing approaches to the design of IASs, and a unified framework was developed to describe these design approaches using consistent and unambiguous terminology. Combining design methodologies from both Human Computer Interaction (HCI) and Human Factors (HF) fields, conceptual and design frameworks were also developed to provide guidelines for the design and implementation of IASs. Finally, a number of criteria that can be used to select appropriate analytical techniques and design approaches were also developed. The proposed frameworks not only provide guidelines for designing IASs in the military domain, but also guide the design of other generic systems to optimize human-machine system performance.

[1]  Joel S. Greenstein,et al.  An Empirical Comparison of Model-Based and Explicit Communication for Dynamic Human-Computer Task Allocation , 1986, Int. J. Man Mach. Stud..

[2]  H. D. Kribs,et al.  Cognitive and Behavioral Task Implications for Three-Dimensional Displays Used in Combat Information/Direction Centers , 1999 .

[3]  Emilie M. Roth,et al.  The Development of a Computer-Aided Cognitive Systems Engineering Tool to Facilitate the Design of Advanced Decision Support Systems , 2001 .

[4]  Pedro M. Domingos,et al.  Mixed initiative interfaces for learning tasks: SMARTedit talks back , 2001, IUI '01.

[5]  Christopher A. Miller,et al.  User acceptance of an intelligent user interface: a Rotorcraft Pilot's Associate example , 1998, IUI '99.

[6]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[7]  Mica R. Endsley,et al.  Situation Awareness Oriented Design: From User's Cognitive Requirements to Creating Effective Supporting Technologies , 2003 .

[8]  William B. Rouse,et al.  Adaptive Allocation of Decisionmaking Responsibility between Human and Computer in Multitask Situations , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Stanley M. Halpin,et al.  Adaptive User Interfaces for Planning and Decision Aids in C3I Systems , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  Arne Worm,et al.  Integrating Scientific Disciplines for Automated Command Support in High-Risk Missions , 2003 .

[11]  Raja Parasuraman,et al.  The Application of a Qualitative Model of Human-Interaction with Automation: Effects of Unreliable Automation on Performance , 2003 .

[12]  B. Trouvain,et al.  Design and Evaluation of a Multi-Robot Control Interface , 2003 .

[13]  David B. Kaber,et al.  Adaptive Automation of Human-Machine System Information-Processing Functions , 2005, Hum. Factors.

[14]  R Parasuraman,et al.  AUTOMATION AND HUMAN PERFORMANCE IN AVIATION. IN: PRINCIPLES AND PRACTICE OF AVIATION PSYCHOLOGY , 2003 .

[15]  John M. Reising,et al.  The Human-Electronic Crew: Is the Team Maturing? Joint GAF/RAF/USAF workshop on Human-Electronic Crew Teamwork (2nd) Held in Ingolstadt, Germany on September 25-28, 1990 , 1992 .

[16]  Kevin B. Bennett,et al.  A Theoretical Analysis and Preliminary Investigation of Dynamically Adaptive Interfaces , 2001 .

[17]  Robert M. Taylor.  Capability, Cognition and Autonomy , 2003 .

[18]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[19]  Christopher A Russell,et al.  Operator State Estimation for Adaptive Aiding in Uninhabited Combat Air Vehicles , 2005 .

[20]  Douglas S. Lange.  Knowledge Base Formation Using Integrated Complex Information , 2001 .

[21]  Erik Hollnagel.  Control Versus Dependence: Striking the Balance in Function Allocation , 1997, HCI.

[22]  Clifford Nass,et al.  Designing social presence of social actors in human computer interaction , 2003, CHI '03.

[23]  Kim J. Vicente,et al.  Comparison of Display Requirements Generated Via Hierarchical Task and Abstraction-Decomposition Space Analysis Techniques , 2001 .

[24]  Frank Lui.  Mapping Cognitive Work Analysis (CWA) To An Intelligent Agents Software Architecture: Command Agents , 2002 .

[25]  Emilie M. Roth,et al.  Work-centered support systems: a human-centered approach to intelligent system design , 2005, IEEE Intelligent Systems.

[26]  Abe Crystal,et al.  Task analysis and human-computer interaction: approaches, techniques, and levels of analysis , 2004, AMCIS.

[27]  Kjeld Schmidt,et al.  Taxonomy for Cognitive Work Analysis , 1990 .

[28]  Nigel Shadbolt,et al.  Cognitive cockpit engineering: Coupling functional state assessment, task knowledge management and decision support for context sensitive aiding , 2001 .

[29]  Morten Grandt,et al.  Human Centered Decision Support for Anti-Air Warfare on Naval Platforms , 2003 .

[30]  Sholom Cohen,et al.  DoD Experience with the C4ISR Architecture Framework , 2003 .

[31]  Robert P. Goldman,et al.  The Playbook™ Approach to Adaptive Automation , 2005 .

[32]  Christopher A. Miller,et al.  Trust in Adaptive Automation : The Role of Etiquette in Tuning Trust via Analogic and Affective Methods , 2005 .

[33]  Wayne Zachary,et al.  Cognitive task analysis and modeling of decision making in complex environments. , 1998 .

[34]  Peter A. Hancock,et al.  Implementing Adaptive Function Allocation , 2001 .

[35]  R. Clark,et al.  Cognitive task analysis for training , 1996 .

[36]  Clare-Marie Karat,et al.  User Attitudes Regarding a User-Adaptive eCommerce Web Site , 2003, User Modeling and User-Adapted Interaction.

[37]  Mark T. Maybury,et al.  Communicative Acts for Explanation Generation , 1992, Int. J. Man Mach. Stud..

[38]  George Cybenko,et al.  Agent-Based Systems Engineering , 2005 .

[39]  Maarten Sierhuis,et al.  Adjustable Autonomy and Human-Agent Teamwork in Practice: An Interim Report on Space Applications , 2003 .

[40]  Raya Fidel,et al.  From information behaviour research to the design of information systems: the Cognitive Work Analysis framework , 2004, Inf. Res..

[41]  John Reising.  Uninhabited Systems and Operator Control , 2005 .

[42]  Charlotte Averman,et al.  Using "Human-in-the-loop" in an Adaptive System: An Evalutation Study of the ConCall System , 1999 .

[43]  Reiner Onken,et al.  Intent and error recognition as part of a knowledge-based cockpit assistant , 1998, Defense, Security, and Sensing.

[44]  M. Bonner,et al.  ADAPTIVE AUTOMATION AND DECISION AIDING IN THE MILITARY FAST JET DOMAIN , 2000 .

[45]  Mitsuru Ishizuka,et al.  Eye movements as indices for the utility of life-like interface agents: A pilot study , 2007, Interact. Comput..

[46]  C D Means,et al.  Applying Cognitive Work Analysis to Time Critical Targeting Functionality , 2004 .

[47]  Susan S. Kirschenbaum.  Uncertainty and Automation , 2003 .

[48]  K. J. Vicente,et al.  Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work , 1999 .

[49]  Rouja Nikolova,et al.  Functional Determination of the Operator State in the Interaction of Humans with Automated Systems , 2003 .

[50]  Debra G. Jones,et al.  Designing for Situation Awareness : An Approach to User-Centered Design , 2003 .

[51]  R. Hutton,et al.  Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands. , 1998, Ergonomics.

[52]  Ming Hou,et al.  Intelligent Adaptive Interfaces for the Control of Multiple UAVs , 2007 .

[53]  R. Onken,et al.  Cognitive Cooperation for the Sake of the Human-Machine Team Effectiveness , 2003 .

[54]  Reinhard Oppermann,et al.  Adaptability: user-initiated individualization , 1994 .

[55]  Philip R. Cohen,et al.  Multimodal Command Interaction: Scientific and Technical , 2003 .

[56]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[57]  Raja Parasuraman,et al.  Three Experiments Examining the Use of Electroencephalogram, Event-Related Potentials, and Heart-Rate Variability for Real- Time Human-Centered Adaptive Automation Design , 2003 .

[58]  B. Brehmer.  Dynamic decision making: human control of complex systems. , 1992, Acta psychologica.

[59]  Carol Angus,et al.  Electronic Warfare in the Fifth Dimension: Human Factors Automation Policies and Strategies for Enhanced Situational Awareness and SEAD Performance , 2003 .

[60]  Bruce L. Digney.  Humans Teaching Learning Machines: Apprentice Systems and Shared Control of Military Vehicles , 2003 .

[61]  Sverrir Olafsson.  DYNAMIC TASK ALLOCATION , 1996 .

[62]  Joanna McGrenere,et al.  A comparison of static, adaptive, and adaptable menus , 2004, CHI.

[63]  Dietrich Manzey,et al.  Misuse of Automated Aids in Process Control: Complacency, Automation Bias and Possible Training Interventions , 2006 .

[64]  Raja Parasuraman,et al.  Operator versus computer control of adaptive automation , 1993 .

[65]  John D. Lee,et al.  Human-Automation Collaboration in Dynamic Mission Planning: A Challenge Requiring an Ecological Approach , 2006 .

[66]  Benjamin A. Clegg,et al.  Dynamic Models of Augmented Cognition , 2004, Int. J. Hum. Comput. Interact..

[67]  Christian D. Schunn,et al.  Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction , 2002, Proc. IEEE.

[68]  William B. Rouse,et al.  An Architecture for Intelligent Interfaces: Outline of an Approach to Supporting Operators of Complex Systems , 1987, Hum. Comput. Interact..

[69]  William B. Rouse,et al.  Adaptive Aiding for Human-Computer Control: Experimental Studies of Dynamic Task Allocation. , 1986 .

[70]  Ming Hou,et al.  19. Operational Analysis and Performance Modeling for the Control of Multiple Uninhabited Aerial Vehicles from an Airborne Platform , 2006 .

[71]  Etienne Tarnowski.  Cockpit Automation Philosophy , 2003 .

[72]  Raja Parasuraman,et al.  18. Delegation Interfaces for Human Supervision of Multiple Unmanned Vehicles: Theory, Experiments, and Practical Applications , 2006 .

[73]  Eduardo Salas,et al.  Analyzing knowledge requirements in team tasks , 2000 .

[74]  Analía Amandi,et al.  Personal assistants: Direct manipulation vs. mixed initiative interfaces , 2006, Int. J. Hum. Comput. Stud..

[75]  Neelam Naikar.  An Examination of the Key Concepts of the Five Phases of Cognitive Work Analysis with Examples from a Familiar System , 2006 .

[76]  W. T. Powers.  Behavior, the control of perception , 1973 .

[77]  A. W. Evans,et al.  A Scale MOUT Facility for Studying Human-Robot Interaction and Control , 2004 .

[78]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[79]  Robert G. Eggleston,et al.  Work Centered Support System Design: Using Frames to Reduce Work Complexity , 2002 .

[80]  Kim J. Vicente,et al.  Ecological interface design: theoretical foundations , 1992, IEEE Trans. Syst. Man Cybern..

[81]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[82]  R. J. Harder,et al.  Application of ThinkLets to team cognitive task analysis , 2004 .

[83]  D. P. Baker,et al.  Measuring the Importance of Teamwork: The Reliability and Validity of Job/Task Analysis Indices for Team-Training Design , 1994 .

[84]  A. B. de Castro,et al.  Hierarchy of controls , 2003 .

[85]  Robert G. Eggleston,et al.  A Framework for Work-Centered Product Evaluation , 2003 .

[86]  Glenn F. Wilson.  Adaptive Aiding Implemented by Psychophysiologically Determined Operator Functional State , 2003 .

[87]  C W Pleydell-Pearce,et al.  Development of a Cognition Monitor , 1999 .

[88]  M.M. Kokar,et al.  Lessons learned from developing SAWA: a situation awareness assistant , 2005, 2005 7th International Conference on Information Fusion.

[89]  Raja Parasuraman,et al.  Human-Automation Interaction , 2005 .

[90]  Pat Langley,et al.  An adaptive stock tracker for personalized trading advice , 2003, IUI '03.

[91]  Gary Klein,et al.  Applied Cognitive Task Analysis (ACTA) Methodology , 1997 .

[92]  John M. Reising.  Uninhabited Military Vehicles: What Is the Role of the Operators? , 2003 .

[93]  Tiffany Frazier,et al.  Dynamic Control and Formal Models of Multi-Agent Interactions and Behaviors , 2003 .

[94]  General , 1970 .

[95]  William C. Elm,et al.  A Pragmatic Cognitive System Engineering Approach to Model Dynamic Human Decision-Making Activities in Intelligent and Automated Systems , 2003 .

[96]  Eloi Bosse,et al.  The Cognitive Costs and Benefits of Automation , 2003 .

[97]  Christopher G. Healey,et al.  MAVEN-SA: Model-Based Automated Visualization for Enhanced Situation Awareness , 2005 .

[98]  Yvette J. Tenney,et al.  Pilot opinions on high level flight deck automation issues: Toward the development of a design philosophy , 1995 .

[99]  Patrick Le Blaye.  A Concept of Flight Execution Monitor (FEM) for Helicopter Pilot Assistance , 2003 .

[100]  Harry B. Funk,et al.  "CONTEXT SENSITIVE" INTERFACE DESIGN , 1997 .

[101]  Designing an Intelligent Agent? Vertical and Horizontal Features Activate Different Face Schemas , 2006 .

[102]  S S Stevens,et al.  HUMAN ENGINEERING FOR AN EFFECTIVE AIR-NAVIGATION AND TRAFFIC-CONTROL SYSTEM, AND APPENDIXES 1 THRU 3 , 1951 .

[103]  R. M. Taylor.  Technologies for Supporting Human Cognitive Control , 2001 .

[104]  J. Morrison,et al.  DARPA Augmented Cognition Technical Integration Experiment (TIE) , 2003 .

[105]  Brian G. Hilburn.  Evaluating Human Interaction with Advanced Air Traffic Management Automation , 2003 .

[106]  David B. Kaber,et al.  On the Design of Adaptive Automation for Complex Systems , 2001 .

[107]  Mustapha Mouloua,et al.  Effects of Adaptive Task Allocation on Monitoring of Automated Systems , 1996, Hum. Factors.

[108]  Matjaz Debevc,et al.  Design and evaluation of an adaptive icon toolbar , 2004, User Modeling and User-Adapted Interaction.

[109]  J. D. Fletcher,et al.  Computer-Based Aids for Learning, Job Performance, and Decision-Making in Military Applications: Emergent Technology and Challenges , 2002 .

[110]  Nadine B. Sarter,et al.  Team Play with a Powerful and Independent Agent: Operational Experiences and Automation Surprises on the Airbus A-320 , 1997, Hum. Factors.

[111]  Magdalena D. Bugajska,et al.  Agent-based Multimodal Interface for Dynamically Autonomous Mobile Robots , 2003 .

[112]  Alan T. Pope,et al.  Physiological Self-Regulation and Adaptive Automation , 2002 .

[113]  J.B.F. van Erp,et al.  Supervising UMVs: Improving Operator Performance through Anticipatory Interface Concepts , 2006 .

[114]  N. Moray,et al.  Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. , 2000, Journal of experimental psychology. Applied.

[115]  Andrew J. Tattersall,et al.  Developing Dynamic Function Allocation for Future Naval Systems , 1997 .

[116]  Zoë P. Lock,et al.  Personalised Briefing Agents to Improve Situational Awareness , 2002, AOIS@AAMAS.

[117]  Schloss Birlinghoven,et al.  Adaptability and Adaptivity in Learning Systems , 1997 .

[118]  Renee Chow,et al.  Applied Comparison between Hierarchical Goal Analysis and Mission, Function and Task Analysis , 2006 .

[119]  David Franklin,et al.  Plan-based interfaces: keeping track of user tasks and acting to cooperate , 2002, IUI '02.

[120]  Robert P. Goldman,et al.  "Tasking" Interfaces for Flexible Interaction with Automa- tion: Keeping the Operator in Control , 1999 .

[121]  David Woods,et al.  Cognitive consequences of clumsy automation on high workload, high consequence human performance , 1991 .

[122]  Alexander Serenko,et al.  Are interface agents scapegoats? Attributions of responsibility in human-agent interaction , 2007, Interact. Comput..

[123]  Wright-Patterson Afb,et al.  Differences in Pilot Automation Philosophies in the US and Russian Air Forces Ground Collision Avoidance Systems , 2002 .

[124]  Richard W. Moss,et al.  Automation in the Cockpit: Who's in Charge? , 1984 .

[125]  Michael J. Young,et al.  Work-centered Support System technology: a new interface client technology for the battlespace infosphere , 2000, Proceedings of the IEEE 2000 National Aerospace and Electronics Conference. NAECON 2000. Engineering Tomorrow (Cat. No.00CH37093).

[126]  Alex Kirlik,et al.  Comparison of display enhancement with intelligent decision-aiding , 1992 .

[127]  Thomas B. Sheridan,et al.  Human and Computer Control of Undersea Teleoperators , 1978 .

[128]  Roderick Murray-Smith,et al.  Pointing without a pointer , 2004, CHI EA '04.

[129]  Lawrence A. H. Liang.  The use of agent-based simulation for cooperative sensing of the battlefield , 2005 .

[130]  Jack L Edwards.  Plans and Goals in a Testbed for Intelligent Aiding , 2000 .

[131]  John D. Lee,et al.  Trust in Automation: Designing for Appropriate Reliance , 2004 .

[132]  Chris Miller.  Using Delegation as an Architecture for Adaptive Automation , 2005 .

[133]  Ben Shneiderman,et al.  Split menus: effectively using selection frequency to organize menus , 1994, TCHI.

[134]  M. Endsley.  Automation and situation awareness. , 1996 .

[135]  Mary L. Cummings,et al.  The Need for Command and Control Instant Message Adaptive Interfaces: Lessons Learned from Tactical Tomahawk Human-in-the-Loop Simulations , 2004, Cyberpsychology Behav. Soc. Netw..

[136]  R. M. Taylor,et al.  Cognitive Quality and Situational Awareness with Advanced Aircraft Attitude Displays , 1990 .

[137]  James A. Mason,et al.  Explicit Models in Intelligent Interface Design , 1988, SGCH.

[138]  E. Patterson,et al.  Cognitive Engineering : Issues in User-Centered System Design , 2000 .

[139]  Jens Rasmussen,et al.  Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering , 1986 .

[140]  Mica R. Endsley,et al.  Using Goal Directed Task Analysis with Army Brigade Officer Teams , 2002 .

[141]  Hall P. Beck,et al.  A Framework of Automation Use , 2001 .

[142]  Iris Davis,et al.  The Importance of Determining Individual Operator Capabilities When Applying Adaptive Aiding , 2006 .

[143]  Mustapha Mouloua,et al.  Adaptive function allocation reduces performance costs of static automation , 1993 .

[144]  Abuse.  Humans and Automation : Use , Misuse , Disuse , , 2008 .

[145]  William B. Rouse,et al.  Adaptive Aiding for Symbiotic Human-Computer Control: Conceptual Model and Experimental Approach , 1985 .

[146]  Winston Bennett,et al.  Behavioral and Cognitive Task Analysis Integration For Assessing Individual and Team Work Activities , 1998 .

[147]  Neville A Stanton,et al.  Hierarchical task analysis: developments, applications, and extensions. , 2006, Applied ergonomics.

[148]  Bob Kobierski,et al.  Hierarchical Goal Analysis and Performance Modelling for the Control of Multiple UAVs/UCAVs from an Airborne Platform. Volume 1 , 2004 .

[149]  William Rouse,et al.  OPTIMAL ADAPTIVE ALLOCATION OF DECISION MAKING RESPONSIBILITY BETWEEN HUMAN AND COMPUTER IN MULTI-TASK SITUATIONS. , 1977 .

[150]  Ronald T. Kessel,et al.  The Roles of Human Operator and Machine in Decision Aid Strategies for Target Detection , 2003 .

[151]  F. Freeman,et al.  A Closed-Loop System for Examining Psychophysiological Measures for Adaptive Task Allocation , 2000, The International journal of aviation psychology.

[152]  N Moray,et al.  Trust, control strategies and allocation of function in human-machine systems. , 1992, Ergonomics.

[153]  David B. Kaber,et al.  The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task , 2004 .

[154]  R. M. Taylor.  Human Electronic Crew Teamwork: Cognitive Requirements for Compatibility and Control with Dynamic Function Allocation , 1997, HCI.

[155]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[156]  Norman D. Geddes.  Associate Systems: A Framework for Human-Machine Cooperation , 1997, HCI.

[157]  Paul Clements,et al.  DoD Architecture Framework and Software Architecture Workshop Report , 2003 .

[158]  Jack L Edwards.  A Generic, Agent-Based Framework for Design and Development of UAV/UCAV Control Systems , 2004 .

[159]  William C. Elm,et al.  Applied Cognitive Work Analysis (ACWA) , 2004 .

[160]  Jens Rasmussen,et al.  Deciding and Doing: Decision Making in Natural Context , 1993 .

[161]  Robert G. Eggleston,et al.  Work-Centered Decision Support , 2003 .

[162]  S. Wood.  Automated Behavior-Based Interaction Customization for Military Command and Control , 2003 .

[163]  Barbara Hayes-Roth,et al.  Making Intelligent Systems Adaptive , 1988 .