Using Model Checking to Explore Checklist-Guided Pilot Behavior

Pilot noncompliance with checklists has been associated with aviation accidents. This noncompliance can be influenced by complex interactions among the checklist, pilot behavior, aircraft automation, device interfaces, and policy, all within the dynamic flight environment. We present a method that uses model checking to evaluate checklist-guided pilot behavior while considering these interactions. We illustrate our approach with a case study of a pilot performing the “Before Landing” checklist. We use our method to explore how different design interventions could impact the safe arming and deployment of spoilers. Results and future research are discussed.

[1]  E. Allen Emerson,et al.  Temporal and Modal Logic , 1991, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[2]  Scott A. Shappell,et al.  A Human Error Approach to Accident Investigation: The Taxonomy of Unsafe Operations , 1997 .

[3]  J. Van Leeuwen,et al.  Handbook of theoretical computer science - Part A: Algorithms and complexity; Part B: Formal models and semantics , 1990 .

[4]  Carl A. Gunter,et al.  In handbook of theoretical computer science , 1990 .

[5]  David O'Hare,et al.  The Shape of Aviation Psychology: A Review of Articles Published in the First 5 Years of The International Journal of Aviation Psychology , 2000 .

[6]  Ellen J. Bass,et al.  Evaluating human-automation interaction using task analytic behavior models, strategic knowledge-based erroneous human behavior generation, and model checking , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[7]  Earl L. Wiener,et al.  Human factors of flight-deck checklists: The normal checklist , 1990 .

[8]  Natarajan Shankar,et al.  The SAL Language Manual , 2003 .

[9]  Ellen J. Bass,et al.  Using task analytic models to visualize model checker counterexamples , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[10]  Alan J. Hu.  Simulation vs. Formal: Absorb What Is Useful; Reject What Is Useless , 2007, Haifa Verification Conference.

[11]  Fabio Paternò,et al.  Preventing user errors by systematic analysis of deviations from the system task model , 2002, Int. J. Hum. Comput. Stud..

[12]  Ellen J. Bass,et al.  Architecture and development environment of a knowledge-based monitor that facilitate incremental knowledge-base development , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[13]  Jeffrey M. Voas.  Fault Injection for the Masses , 1997, Computer.

[14]  Asaf Degani,et al.  Procedures in complex systems: the airline cockpit , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[15]  Key Dismukes,et al.  The Challenge of Aviation Emergency and Abnormal Situations , 2005 .

[16]  S. Greenberg,et al.  The Psychology of Everyday Things , 2012 .

[17]  Michael Heymann,et al.  Formal Aspects of Procedures: The Problem of Sequential Correctness , 1999 .

[18]  Gabrielle de Brito.  Towards a model for the study of written procedure following in dynamic environments , 2002, Reliab. Eng. Syst. Saf..

[19]  Asaf Degani,et al.  Cockpit Checklists: Concepts, Design, and Use , 1993 .

[20]  Nadine B. Sarter,et al.  Error Types and Related Error Detection Mechanisms in the Aviation Domain: An Analysis of Aviation Safety Reporting System Incident Reports , 2000 .

[21]  Ellen J. Bass,et al.  Formally verifying human–automation interaction as part of a system model: limitations and tradeoffs , 2010, Innovations in Systems and Software Engineering.

[22]  J. Shaoul.  Human Error , 1973, Nature.

[23]  Matthew L Bolton,et al.  A Method for the Formal Verification of Human-interactive Systems. , 2009, Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting. Human Factors and Ergonomics Society. Annual Meeting.

[24]  Robert L. Wears,et al.  Resilience Engineering: Concepts and Precepts , 2006, Quality and Safety in Health Care.

[25]  Karen M. Feigh,et al.  Toward a multi-method approach to formalizing human-automation interaction and human-human communications , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[26]  Jeannette M. Wing.  A specifier's introduction to formal methods , 1990, Computer.

[27]  Asaf Degani,et al.  Taming HAL: Designing Interfaces Beyond 2001 , 2004 .

[28]  Philippe A. Palanque,et al.  Formal modelling of incidents and accidents as a means for enriching training material for satellite control operations , 2008 .

[29]  Ellen J. Bass,et al.  A Systematic Approach to Model Checking Human–Automation Interaction Using Task Analytic Models , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[30]  Robert E. Fields,et al.  Analysis of erroneous actions in the design of critical systems , 2001 .

[31]  Nadine B. Sarter,et al.  How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control , 1995, Hum. Factors.

[32]  Sandra Basnyat,et al.  Error Patterns: Systematic Investigation of Deviations in Task Models , 2006, TAMODIA.

[33]  Erik Hollnagel,et al.  The Phenotype of Erroneous Actions , 1993, Int. J. Man Mach. Stud..

[34]  Immanuel Barshi,et al.  Emergency and Abnormal Situations: a Review of Asrs Reports , 2003 .

[35]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[36]  Matthew L Bolton,et al.  Using Task Analytic Models and Phenotypes of Erroneous Human Behavior to Discover System Failures Using Model Checking , 2010, Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting. Human Factors and Ergonomics Society. Annual Meeting.