Naval aviation squadron risk analysis predictive Bayesian Network Modeling using Maintenance Climate Assessment Survey results

NAVAL AVIATION SQUADRON RISK ANALYSIS PREDICTIVE BAYESIAN NETWORK MODELING USING MAINTENANCE CLIMATE ASSESSMENT SURVEY RESULTS Harry Michael Robinson Old Dominion University, 2014 Director: Dr. John A. Sokolowski Associated risks in flying have resulted in injury or death to aircrew and passengers, and damage or destruction o f the aircraft and its surroundings. Although the Naval Aviation's flight mishap rate declined over the past 60 years, the proportion of human error causal factors has stayed relatively constant at about 80%. Efforts to reduce human errors have focused attention on understanding the aircrew and maintenance actions occurring in complex systems. One such tool has been the Naval Aviation squadrons’ regular participation in survey questionnaires deigned to measure respondent ratings related to personal judgments or perceptions o f organizational climate for meeting the extent to which a particular squadron achieved the High Reliability Organization (HRO) criteria of achieving safe and reliable operations and maintenance practices while working in hazardous environments. Specifically, the Maintenance Climate Assessment Survey (MCAS) is completed by squadron maintainers to enable leadership to assess their unit’s aggregated responses against those from other squadrons. Bayesian Network Modeling and Simulation provides a potential methodology to represent the relationships o f MCAS results and mishap occurrences that can be used to derive and calculate probabilities o f incurring a future mishap. Model development and simulation analysis was conducted to research a causal relationship through quantitative analysis o f conditional probabilities based upon observed evidence o f previously occurred mishaps. This application would enable Navy and Marine Corps aviation squadron leadership to identify organizational safety risks, apply focused proactive measures to mitigate related hazards characterized by the MCAS results, and reduce organizational susceptibility to future aircraft mishaps. This dissertation is dedicated to my family, especially Anne and Andrew, for their support and understanding. I am grateful to my father, Captain “Fast Eddie” Robinson, U.S. Naval Reserve, who introduced me to Naval Aviation and my mother, Deborah Lowenthal Robinson who pinned on my wings o f gold. The research conducted and detailed here within is for the men and women who ardently serve in the United States Navy and Marine Corps. Their determination and spirit is the bedrock of Naval Aviation which enables the capability to

[1]  Scott A. Shappell,et al.  A HUMAN ERROR APPROACH TO AVIATION ACCIDENT ANALYSIS , 2003 .

[2]  Angela O’Dea,et al.  An Evaluation of the Construct Validity of the Command Safety Assessment Survey , 2010 .

[3]  M. Apostolides,et al.  Statistical Thinking for Managers , 1994 .

[4]  Robert G. Sargent,et al.  Validating simulation models , 1983, WSC '83.

[5]  Simon Jackman,et al.  Bayesian Analysis for the Social Sciences , 2009 .

[6]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[7]  R. Likert “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[8]  J. L. Rogers,et al.  Using significance tests to evaluate equivalence between two experimental groups. , 1993, Psychological bulletin.

[9]  J Reason,et al.  The contribution of latent human failures to the breakdown of complex systems. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[10]  Allan H. Church,et al.  Designing and using organizational surveys : a seven-step process , 2001 .

[11]  Ashley D. Fry,et al.  Modeling and analysis of human error in Naval Aviation maintenance mishaps. , 2000 .

[12]  Ann Williamson,et al.  Associations between Errors and Contributing Factors in Aircraft Maintenance , 2003, Hum. Factors.

[13]  A. Parasuraman,et al.  SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. , 1988 .

[14]  Mark Henry Adamshick,et al.  Leadership and Safety Climate in High-Risk Military Organizations , 2007 .

[15]  Nancy G. Leveson,et al.  A New Approach To System Safety Engineering , 2005 .

[16]  Kangsun Lee,et al.  Dynamic model abstraction , 1996, Proceedings Winter Simulation Conference.

[17]  John M. Carroll,et al.  Mental Models in Human-Computer Interaction , 1988 .

[18]  Alison E. Hernandez Organizational Climate and Its Relationship With Aviation Maintenance Safety , 2001 .

[19]  Paul O'Connor,et al.  Identifying and addressing the limitations of safety climate surveys. , 2011, Journal of safety research.

[20]  Barbara G. Kanki,et al.  Patterns of Error in Confidential Maintenance Incident Reports , 2008 .

[21]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model , 1992 .

[22]  Scott D. Sagan,et al.  Learning from Normal Accidents , 2004 .

[23]  Peter Haddawy,et al.  An Overview of Some Recent Developments in Bayesian Problem-Solving Techniques , 1999, AI Mag..

[24]  Karl E. Weick,et al.  Managing the unexpected: Assuring high performance in an age of complexity. , 2001 .

[25]  F. Guldenmund The nature of safety culture: a review of theory and research , 2000 .

[26]  Scott A. Shappell,et al.  The Human Factors Analysis and Classification System : HFACS : final report. , 2000 .

[27]  Kim J. Vicente,et al.  The Walkerton E. coli outbreak: a test of Rasmussen's framework for risk management in a dynamic society , 2006 .

[28]  James T. Luxhoj,et al.  A RISK-BASED DECISION SUPPORT TOOL FOR EVALUATING AVIATION TECHNOLOGY INTEGRATION IN THE NATIONAL AIRSPACE SYSTEM , 2003 .

[29]  Thomas J. Dormody,et al.  Analyzing Data Measured by Individual Likert-Type Items. , 1994 .

[30]  J Rasmussen,et al.  Human error and the problem of causality in analysis of accidents. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[31]  David C Krulak,et al.  Human factors in maintenance: impact on aircraft mishap frequency and severity. , 2004, Aviation, space, and environmental medicine.

[32]  Scott A. Shappell,et al.  A Human Error Analysis of Commercial Aviation Accidents Using the Human Factors Analysis and Classification System (HFACS) , 2001 .

[33]  Edward B. Fowlkes,et al.  Risk analysis of the space shuttle: Pre-Challenger prediction of failure , 1989 .

[34]  L. Daley,et al.  Intra-Industry Effects of the Accident at Three Mile Island , 1983, Journal of Financial and Quantitative Analysis.

[35]  James T. Reason,et al.  Managing the risks of organizational accidents , 1997 .

[36]  C.A.L. Bailer-Jones,et al.  An introduction to artificial neural networks , 2001 .

[37]  M. F. Ramalhoto,et al.  Ordinal Methodology in the Analysis of Likert Scales , 2007 .

[38]  Simon Bennett Managing Maintenance Error: A Practical Guide , 2004 .

[39]  J. Gliem,et al.  Calculating, Interpreting, And Reporting Cronbach’s Alpha Reliability Coefficient For Likert-Type Scales , 2003 .

[40]  Nancy G. Leveson,et al.  Beyond Normal Accidents and High Reliability Organizations: The Need for an Alternative Approach to Safety in Complex Systems , 2004 .

[41]  Josh Tabor,et al.  Investigating the Investigative Task: Testing for Skewness , 2010 .

[42]  D. L. Simms,et al.  Normal Accidents: Living with High-Risk Technologies , 1986 .

[43]  S. Albin,et al.  Building a System Dynamics Model Part 1: Conceptualization , 1997 .

[44]  Nancy G. Leveson,et al.  Safety and Risk -Driven Design in Complex Systems -of - Systems , 2005 .

[45]  Gideon Vigderhous,et al.  The Level of Measurement and “Permissible” Statistical Analysis in Social Research , 1977 .

[46]  K. Roberts Managing High Reliability Organizations , 1990 .

[47]  Michelle Ryerson,et al.  Use of Human Factors Analysis for Wildland Fire Accident Investigations , 2005 .

[48]  L. Cronbach Test “reliability”: Its meaning and determination , 1947, Psychometrika.

[49]  Hessam S. Sarjoughian,et al.  Multi-formalism modeling approach for semiconductor supply/demand networks , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[50]  I. M. Crawford Marketing Research and Information Systems , 1997 .

[51]  Paul T. von Hippel,et al.  Mean, Median, and Skew: Correcting a Textbook Rule , 2005 .

[52]  Anthony P. Ciavarelli HFES 51st Annual Meeting: Assessing Safety Climate and Organizational Risk , 2007 .

[53]  James Reason,et al.  Human Error , 1990 .

[54]  Jun-Jang Jeng,et al.  On developing system dynamics model for business process simulation , 2005, Proceedings of the Winter Simulation Conference, 2005..

[55]  Dylan D. Schmorrow,et al.  A Human Error Analysis and Model of Naval Aviation Maintenance Related Mishaps. , 1998 .

[56]  Katherine A. Berry,et al.  A meta-analysis of Human Factors Analysis and Classification System causal factors: Establishing benchmarking standards and human error latent failure pathway associations in various domains , 2010 .

[57]  Yves Boussemart,et al.  Predictive Models of Procedural Human Supervisory Control Behavior , 2011 .

[58]  Michael E. Fischer Mission-Type Orders in Joint Air Operations: The Empowerment of Air Leadership , 2012 .

[59]  Nancy G. Leveson,et al.  An analysis of causation in aerospace accidents , 2001, 20th DASC. 20th Digital Avionics Systems Conference (Cat. No.01CH37219).

[60]  H. Kraemer,et al.  Summated Rating Scale Construction An Introduction , 2009 .

[61]  John M Linacre,et al.  Optimizing rating scale category effectiveness. , 2002, Journal of applied measurement.

[62]  David C. Howell,et al.  Unidimensional Scaling , 2004 .

[63]  Kathryn Mearns,et al.  Measuring safety climate: identifying the common features☆ , 2000 .

[64]  John P. Snow,et al.  Integrating naval aviation maintenance training with sailors' proficiency qualification and its correlation with improved weapon system readiness , 2007 .

[65]  Yaneer Bar-Yam,et al.  Dynamics Of Complex Systems , 2019 .

[66]  Brent W. Goodrum Assessment of maintenance safety climate in U.S. Navy Fleet Logistics Support Wing Squadrons , 1999 .

[67]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[68]  Robert G. Sargent,et al.  Some approaches and paradigms for verifying and validating simulation models , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[69]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[70]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[71]  K. Roberts Some Characteristics of One Type of High Reliability Organization , 1990 .

[72]  Rouslan A. Moro,et al.  Support Vector Machines (SVM) as a Technique for Solvency Analysis , 2008 .

[73]  D. Pace Ideas About Simulation Conceptual Model Development , 2000 .

[74]  David Heckerman,et al.  Learning With Bayesian Networks (Abstract) , 1995, ICML.

[75]  M. C. Hart Improving the dissemination of SERVQUAL by using magnitude scaling , 1996 .

[76]  Dylan D. Schmorrow,et al.  Human Factors Analysis of Naval Aviation Maintenance Related Mishaps , 2000 .

[77]  A. Parasuraman,et al.  More on improving service quality measurement , 1993 .

[78]  R. Green Human error on the flight deck. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[79]  J. DeCoster Scale Construction Notes , 2005 .

[80]  David C. Nagel,et al.  Human Error in Aviation Operations , 1988 .

[81]  Paul R. Garvey,et al.  Analytical Methods for Risk Management: A Systems Engineering Perspective , 2008 .

[82]  K. Weick The Vulnerable System: An Analysis of the Tenerife Air Disaster , 1990 .

[83]  Pedro M. Domingos,et al.  Naive Bayes models for probability estimation , 2005, ICML.

[84]  Joseph H. Saleh,et al.  Archetypes for organizational safety , 2006 .

[85]  Scott A. Shappell,et al.  Examining Error Pathways: An Analysis of Contributing Factors using HFACS in Non-Aviation Industries , 2010 .

[86]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[87]  Upendra Dave,et al.  Probabilistic Reasoning and Bayesian Belief Networks , 1996 .

[88]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[89]  W. Kadous GRASP: Recognition of Australian Sign Language Using Instrumented Gloves , 1995 .

[90]  Prasad V. Prabhu,et al.  Human Error and User-Interface Design , 1997 .

[91]  Scott D. Sagan,et al.  The Problem of Redundancy Problem: Why More Nuclear Security Forces May Produce Less Nuclear Security † , 2004, Risk analysis : an official publication of the Society for Risk Analysis.

[92]  Lakhmi C. Jain,et al.  Introduction to Bayesian Networks , 2008 .

[93]  S. Jamieson Likert scales: how to (ab)use them , 2004, Medical education.

[94]  Nancy G. Leveson,et al.  A new accident model for engineering safer systems , 2004 .

[95]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[96]  James T. Luxhoj Probabilistic Causal Analysis for System Safety Risk Assessments in Commercial Air Transport , 2003 .

[97]  Robert B. Allen,et al.  Mental Models and User Models , 1997 .

[98]  Osman Balci Principles and techniques of simulation validation, verification, and testing , 1995, WSC '95.

[99]  Michael J. Prietula,et al.  Simulating organizations: computational models of institutions and groups , 1998 .

[100]  Norman E. Wallen,et al.  How to Design and Evaluate Research in Education , 1990 .

[101]  Darren George,et al.  SPSS for Windows Step by Step: A Simple Guide and Reference , 1998 .