Identifying Problems and Generating Recommendations for Enhancing Complex Systems: Applying the Abstraction Hierarchy Framework as an Analytical Tool

Objective: This study adopts J. Rasmussen's (1985) abstraction hierarchy (AH) framework as an analytical tool to identify problems and pinpoint opportunities to enhance complex systems. Background: The process of identifying problems and generating recommendations for complex systems using conventional methods is usually conducted based on incompletely defined work requirements. As the complexity of systems rises, the sheer mass of data generated from these methods becomes unwieldy to manage in a coherent, systematic form for analysis. There is little known work on adopting a broader perspective to fill these gaps. Method: AH was used to analyze an aircraft-automation system in order to further identify breakdowns in pilot-automation interactions. Four steps follow: developing an AH model for the system, mapping the data generated by various methods onto the AH, identifying problems based on the mapped data, and presenting recommendations. Results: The breakdowns lay primarily with automation operations that were more goal directed. Identified root causes include incomplete knowledge content and ineffective knowledge structure in pilots' mental models, lack of effective higher-order functional domain information displayed in the interface, and lack of sufficient automation procedures for pilots to effectively cope with unfamiliar situations. Conclusion: The AH is a valuable analytical tool to systematically identify problems and suggest opportunities for enhancing complex systems. It helps further examine the automation awareness problems and identify improvement areas from a work domain perspective. Application: Applications include the identification of problems and generation of recommendations for complex systems as well as specific recommendations regarding pilot training, flight deck interfaces, and automation procedures.

[1]  Nadine B. Sarter,et al.  Pilot Interaction With Cockpit Automation: Operational Experiences With the Flight Management System , 1992 .

[2]  Barry Kirwan,et al.  A Guide To Task Analysis: The Task Analysis Working Group , 1992 .

[3]  Ann M. Bisantz,et al.  Modeling Automation Within an Abstraction Hierarchy , 2003 .

[4]  Catherine M. Burns,et al.  Mental Models and Ecological Interface Design: An Experimental Investigation , 2002 .

[5]  Wei Xu,et al.  Facilitate Complex Search Tasks in Hypertext by Externalizing Functional Properties of a Work Domain , 1999, Int. J. Hum. Comput. Interact..

[6]  John R. Hajdukiewicz,et al.  A theoretical note on the relationship between work domain analysis and task analysis , 2004 .

[7]  Penelope M. Sanderson,et al.  Testing the Impact of Instrumentation Location and Reliability on Ecological Interface Design: Control Performance , 2000 .

[8]  Randall J. Mumaw,et al.  Modern Flight Deck Automation: Pilots’ Mental Model and Monitoring Patterns and Performance , 2003 .

[9]  Nadine B. Sarter,et al.  Team Play with a Powerful and Independent Agent: Operational Experiences and Automation Surprises on the Airbus A-320 , 1997, Hum. Factors.

[10]  Jens Rasmussen,et al.  The role of hierarchical knowledge representation in decisionmaking and system management , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  Raja Parasuraman,et al.  Human-Automation Interaction , 2005 .

[12]  Kim J. Vicente,et al.  Ecological interface design: theoretical foundations , 1992, IEEE Trans. Syst. Man Cybern..

[13]  Christopher D. Wickens,et al.  Pilots' Monitoring Strategies and Performance on Automated Flight Decks: An Empirical Study Combining Behavioral and Eye-Tracking Data , 2007, Hum. Factors.

[14]  H. Schneider.  Failure mode and effect analysis : FMEA from theory to execution , 1996 .

[15]  Kim J. Vicente,et al.  Improving dynamic decision making in complex systems through ecological interface design: A research overview , 1996 .

[16]  Kim J. Vicente,et al.  Supporting operator problem solving through ecological interface design , 1995, IEEE Trans. Syst. Man Cybern..

[17]  K. J. Vicente,et al.  Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work , 1999 .

[18]  Asaf Degani,et al.  Formal Verification of Human-Automation Interaction , 2002, Hum. Factors.

[19]  Penelope M. Sanderson,et al.  Work Domain Analysis for Training-System Definition and Acquisition , 1999 .

[20]  P.J. Sallis,et al.  The ecological approach to interface design: applying the abstraction hierarchy to intentional domains , 1998, Proceedings 1998 Australasian Computer Human Interaction Conference. OzCHI'98 (Cat. No.98EX234).

[21]  Penelope M. Sanderson,et al.  Evaluating Design Proposals for Complex Systems with Work Domain Analysis , 2001, Hum. Factors.

[22]  Christopher D. Wickens,et al.  ANALYSIS OF PILOTS' MONITORING AND PERFORMANCE ON AN AUTOMATED FLIGHT DECK , 2001 .

[23]  Edwin Hutchins,et al.  The Integrated Mode Management Interface , 1996 .

[24]  Kim J. Vicente,et al.  Designing Functional Visualizations for Aircraft Systems Status Displays , 1999 .

[25]  Kim J. Vicente,et al.  Designing Effective Human-Automation-Plant Interfaces: A Control-Theoretic Perspective , 2005, Hum. Factors.

[26]  A J Belyavin,et al.  Pilots' Attitudes to Cockpit Automation , 1991 .

[27]  Nancy G. Leveson.  Intent Specifications: An Approach to Building Human-Centered Specifications , 2000, IEEE Trans. Software Eng..

[28]  Catherine M. Burns,et al.  Ecological Interface Design in Aviation Domains: Work Domain Analysis of Automated Collision Detection and Avoidance , 2003 .

[29]  Kim J. Vicente,et al.  Ecological Interface Design: Progress and Challenges , 2002, Hum. Factors.

[30]  Catherine M. Burns,et al.  Work Domain Analysis for Intentional Systems , 1999 .

[31]  Nadine B. Sarter,et al.  Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study , 2000, Hum. Factors.

[32]  Nadine B. Sarter,et al.  Pilot Interaction With Cockpit Automation II: An Experimental Study of Pilots’ Model and Awareness of the Flight Management System , 1994 .

[33]  Penelope M. Sanderson,et al.  Designing Teams for First-of-a-Kind, Complex Systems Using the Initial Phases of Cognitive Work Analysis: Case Study , 2003, Hum. Factors.

[34]  Penelope M. Sanderson,et al.  Testing the Impact of Instrumentation Location and Reliability on Ecological Interface Design: Fault Diagnosis Performance , 2000 .

[35]  Jens Rasmussen,et al.  Cognitive Systems Engineering , 2022 .

[36]  Catherine M. Burns,et al.  Ecological Interface Design , 2004 .

[37]  Nadine B. Sarter,et al.  How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control , 1995, Hum. Factors.