Failure Assessment

Failure assessment encompasses the identification and characterization both of potential failure mechanisms in systems under development and of actual failure occurrences in operational systems. This paper presents several of the most widely used and useful techniques for failure assessment across the system lifecycle with an emphasis on the role of software. For each technique the paper describes its purpose and background, summarizes the process of performing the technique, and evaluates the technique’s strengths and limitations. The discussion provides lessons learned from practice, examples from spacecraft applications, and pointers to additional work in the field. The paper describes some of the tools that are available to help the practitioner select and implement failure assessment techniques and identifies likely future directions in failure assessment.

[1]  Donald J. Reifer,et al.  Software Failure Modes and Effects Analysis , 1979, IEEE Transactions on Reliability.

[2]  Norman F. Schneidewind,et al.  Applying reliability models to the space shuttle , 1992, IEEE Software.

[3]  Norman F. Schneidewind Investigation of the risk to software reliability and maintainability of requirements changes , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[4]  Enrico Gobbetti,et al.  Encyclopedia of Electrical and Electronics Engineering , 1999 .

[5]  John C. Munson,et al.  Finding Fault with Faults: A Case Study , 1997 .

[6]  Standard Glossary of Software Engineering Terminology , 1990 .

[7]  Aditya P. Mathur,et al.  Comparison of architecture-based software reliability models , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[8]  J. Wolfowitz,et al.  An Introduction to the Theory of Statistics , 1951, Nature.

[9]  Z. Jelinski,et al.  Software reliability Research , 1972, Statistical Computer Performance Evaluation.

[10]  Katerina Goseva-Popstojanova,et al.  Architecture-based approach to reliability assessment of software systems , 2001, Perform. Evaluation.

[11]  Qian Feng,et al.  Bi-directional safety analysis of product lines , 2005, J. Syst. Softw..

[12]  David R. Cox,et al.  The statistical analysis of series of events , 1966 .

[13]  Swapna S. Gokhale,et al.  Reliability simulation of component-based software systems , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[14]  Peter Neumann,et al.  Safeware: System Safety and Computers , 1995, SOEN.

[15]  John C. Munson,et al.  The effects of fault counting methods on fault model quality , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[16]  Michael R. Lyu,et al.  An experiment in determining software reliability model applicability , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[17]  Robyn R. Lutz,et al.  Applying adaptive safety analysis techniques [for embedded software] , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[18]  David Coppit,et al.  Developing a high-quality software tool for fault tree analysis , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[19]  N. Schneidewind A Recommended Practice for Software Reliability , 2022 .

[20]  A. P. Nikora,et al.  A practical software fault measurement and estimation framework , 2001 .

[21]  Vasant Honavar,et al.  A Software Fault Tree Approach to Requirements Analysis of an Intrusion Detection System , 2002, Requirements Engineering.

[22]  Bev Littlewood,et al.  Evaluation of competing software reliability predictions , 1986, IEEE Transactions on Software Engineering.

[23]  Ann Marie Neufelder,et al.  Ensuring Software Reliability , 1992 .

[24]  John C. Munson,et al.  Determining fault insertion rates for evolving software systems , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[25]  Norman F. Schneidewind,et al.  Successful application of software reliability engineering for the NASA Space Shuttle , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[26]  Josh Dehlinger,et al.  PLFaultCAT: A Product-Line Software Fault Tree Analysis Tool , 2006, Automated Software Engineering.

[27]  Neil R. Storey,et al.  Safety-critical computer systems , 1996 .

[28]  Jaroslav Menčík,et al.  Failure Modes and Effects Analysis , 2016 .

[29]  Robyn R. Lutz,et al.  Requirements analysis using forward and backward search , 1997, Ann. Softw. Eng..

[30]  John C. Munson,et al.  Developing fault predictors for evolving software systems , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[31]  Glenn Reeves,et al.  Software architecture themes in JPL's Mission Data System , 1999, 2000 IEEE Aerospace Conference. Proceedings (Cat. No.00TH8484).

[32]  G. F. Williamson,et al.  Software safety and reliability , 1997 .

[33]  John D. Musa,et al.  Software Reliability Engineering: More Reliable Software Faster and Cheaper , 2004 .

[34]  Anders P. Ravn,et al.  From Safety Analysis to Software Requirements , 1998, IEEE Trans. Software Eng..

[35]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[36]  Martin L. Shooman Probabilistic Models for Software reliability Prediction , 1972, Statistical Computer Performance Evaluation.

[37]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[38]  John C. Munson,et al.  Understanding the nature of software evolution , 2003, International Conference on Software Maintenance, 2003. ICSM 2003. Proceedings..

[39]  Taghi M. Khoshgoftaar,et al.  The use of software complexity metrics in software reliability modeling , 1991, Proceedings. 1991 International Symposium on Software Reliability Engineering.

[40]  Franklin A. Graybill,et al.  Introduction to the Theory of Statistics, 3rd ed. , 1974 .

[41]  Josh Dehlinger,et al.  Software fault tree analysis for product lines , 2004, Eighth IEEE International Symposium on High Assurance Systems Engineering, 2004. Proceedings..