Comparison of the FMEA and STPA safety analysis methods–a case study

As our society becomes more and more dependent on IT systems, failures of these systems can harm more and more people and organizations. Diligently performing risk and hazard analysis helps to minimize the potential harm of IT system failures on the society and increases the probability of their undisturbed operation. Risk and hazard analysis is an important activity for the development and operation of critical software intensive systems, but the increased complexity and size puts additional requirements on the effectiveness of risk and hazard analysis methods. This paper presents a qualitative comparison of two hazard analysis methods, failure mode and effect analysis (FMEA) and system theoretic process analysis (STPA), using case study research methodology. Both methods have been applied on the same forward collision avoidance system to compare the effectiveness of the methods and to investigate what are the main differences between them. Furthermore, this study also evaluates the analysis process of both methods by using a qualitative criteria derived from the technology acceptance model (TAM). The results of the FMEA analysis were compared to the results of the STPA analysis, which were presented in a previous study. Both analyses were conducted on the same forward collision avoidance system. The comparison shows that FMEA and STPA deliver similar analysis results.

[1]  Michael Felderer,et al.  Testing Security Requirements with Non-experts: Approaches and Empirical Investigations , 2016, 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[2]  Michael R. Beauregard,et al.  The Basics of FMEA , 1996 .

[3]  Jiwen Liu,et al.  A comparison of FMEA, AFMEA and FTA , 2011, The Proceedings of 2011 9th International Conference on Reliability, Maintainability and Safety.

[4]  John A. McDermid,et al.  Experience with the application of HAZOP to computer-based systems , 1995, COMPASS '95 Proceedings of the Tenth Annual Conference on Computer Assurance Systems Integrity, Software Safety and Process Security'.

[5]  Steven J Pereira,et al.  A System-Theoretic Hazard Analysis Methodology for a Non-advocate Safety Assessment of the Ballistic Missile Defense System , 2006 .

[6]  Christine Nadel,et al.  Case Study Research Design And Methods , 2016 .

[7]  Felix Redmill,et al.  Lessons in System Safety , 1999 .

[8]  Martin Höst,et al.  Comparison of the FMEA and STPA safety analysis methods–a case study , 2017, Software quality journal.

[9]  Andreas L. Opdahl,et al.  Eliciting security requirements with misuse cases , 2004, Requirements Engineering.

[10]  Cody H. Fleming,et al.  Safety assurance in NextGen and complex transportation systems , 2013 .

[11]  Nancy G. Leveson,et al.  Engineering a Safer World: Systems Thinking Applied to Safety , 2012 .

[12]  Cody H. Fleming,et al.  Safety Assurance in NextGen , 2012 .

[13]  Donald J. Reifer,et al.  Software Failure Modes and Effects Analysis , 1979, IEEE Transactions on Reliability.

[14]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[15]  Sylwia Męcfal.  Recenzja książki. Robert K. yin, Case Study Research. Design and Methods (fourth Edition), thousand Oaks, CA: Sage Publications, 2009 , 2012 .

[16]  Nancy G. Leveson,et al.  Safety Assessment of Complex, Software-Intensive Systems , 2012 .

[17]  Martin Höst,et al.  A review of research on risk analysis methods for IT systems , 2013, EASE '13.

[18]  H. Schneider.  Failure mode and effect analysis : FMEA from theory to execution , 1996 .

[19]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[20]  James Carlin Becker,et al.  A practical approach to failure mode, effects and criticality analysis (FMECA) for computing systems , 1996, Proceedings. IEEE High-Assurance Systems Engineering Workshop (Cat. No.96TB100076).

[21]  Guido Menkhaus,et al.  Metric Suite Directing the Failure Mode Analysis of Embedded Software Systems , 2005, ICEIS.

[22]  Kosuke Ishii,et al.  ADVANCED FAILURE MODES AND EFFECTS ANALYSIS USING BEHAVIOR MODELING , 1997 .

[23]  Austen Rainer,et al.  Case Study Research in Software Engineering - Guidelines and Examples , 2012 .

[24]  Mattias Bengtsson,et al.  Collision Warning with Full Auto Brake and Pedestrian Detection - a practical example of Automatic Emergency Braking , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[25]  Martin Höst,et al.  Hazard analysis of collision avoidance system using STPA , 2014, ISCRAM.

[26]  Felix Redmill,et al.  System Safety: HAZOP and Software HAZOP , 1999 .

[27]  Nancy G. Leveson,et al.  Safety Guided Design of Crew Return Vehicle in Concept Design Phase Using Stamp / Stpa , 2011 .

[28]  Andreas L. Opdahl,et al.  Comparing risk identification techniques for safety and security requirements , 2013, J. Syst. Softw..

[29]  Ina Schieferdecker,et al.  A taxonomy of risk-based testing , 2014, International Journal on Software Tools for Technology Transfer.

[30]  Christoph Schmittner,et al.  Security Application of Failure Mode and Effect Analysis (FMEA) , 2014, SAFECOMP.

[31]  Christine M. Haissig,et al.  Using TCAS surveillance to enable legacy ADS-B transponder use for in-trail procedures , 2012, 2012 IEEE/AIAA 31st Digital Avionics Systems Conference (DASC).

[32]  Kim H. Pries.  Failure Mode & Effects Analysis in Software Development , 1998 .

[33]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[34]  Yan Li,et al.  Approaches for the combined use of risk analysis and testing: a systematic literature review , 2014, International Journal on Software Tools for Technology Transfer.

[35]  John Thomas,et al.  Modeling and Hazard Analysis Using Stpa , 2010 .

[36]  Tor Stålhane,et al.  A Comparison of Two Approaches to Safety Analysis Based on Use Cases , 2007, ER.

[37]  Michael Felderer,et al.  A Process for Risk-Based Test Strategy Development and Its Industrial Evaluation , 2015, PROFES.

[38]  D. Hickey.  Distritrack: Automated Average-Case Analysis , 2007 .

[39]  Nancy G. Leveson,et al.  A systems-theoretic approach to safety in software-intensive systems , 2004, IEEE Transactions on Dependable and Secure Computing.

[40]  Ross T. Smith,et al.  The Practical Guide to Defect Prevention , 2007 .

[41]  Felix Redmill,et al.  Risk analysis-a subjective process , 2002 .

[42]  Nancy G. Leveson,et al.  Hazard Analysis of Complex Spacecraft Using Systems-Theoretic Process Analysis , 2014 .

[43]  J. Thomas.  Performing Hazard Analysis on Complex, Software- and Human-Intensive Systems , 2011 .

[44]  Lars Grunske,et al.  Probabilistic Model-Checking Support for FMEA , 2007, Fourth International Conference on the Quantitative Evaluation of Systems (QEST 2007).