Fail-safe testing of safety-critical systems: a case study and efficiency analysis

This paper proposes an approach for testing of safety-critical systems. It is based on a behavioral and a fault model. The two models are analyzed for compatibility, and necessary changes are identified to make them compatible. Then, transformation rules are used to transform the fault model into the same model type as the behavioral model. Integration rules define how to combine them. This approach results in an integrated model which then can be used to generate tests using a variety of testing criteria. The paper illustrates this general framework using a CEFSM for the behavioral model and a fault tree for the fault model. We apply the technique to an aerospace launch system. We also investigate the scalability of the approach and compare its efficiency with integrating a state chart and a fault tree.

[1]  Danhua Wang,et al.  An optimization to automatic Fault Tree Analysis and Failure Mode and Effect Analysis approaches for processes , 2010, 2010 International Conference On Computer Design and Applications.

[2]  Carol S. Smidts,et al.  An experimental evaluation of a higher-ordered-typed-functional specification-based test-generation technique , 2006, Empirical Software Engineering.

[3]  Paul Pettersson,et al.  A Global Algorithm for Model-Based Test Suite Generation , 2007, Electron. Notes Theor. Comput. Sci..

[4]  Kerstin Buchacker,et al.  Combining Fault Trees And Petri Nets To Model Safety-Critical Systems , 1999 .

[5]  Radu Mateescu,et al.  Large-scale Distributed Verification Using CADP: Beyond Clusters to Grids , 2013, Electron. Notes Theor. Comput. Sci..

[6]  Dieter Hogrefe,et al.  Towards the industrial use of validation techniques and automatic test generation methods for SDL specifications , 1997, SDL Forum.

[7]  Stefano Marrone,et al.  Towards Model-Driven V&V assessment of railway control systems , 2014, International Journal on Software Tools for Technology Transfer.

[8]  Francesco Liberati,et al.  Interdependency modeling and analysis of critical infrastructures based on Dynamic Bayesian Networks , 2011, 2011 19th Mediterranean Conference on Control & Automation (MED).

[9]  Rachida Dssouli,et al.  A Guided Incremental Test Case Generation Procedure for Conformance Testing for CEFSM Specified Protocols , 1998, IWTCS.

[10]  Jan Tretmans,et al.  Model Based Testing with Labelled Transition Systems , 2008, Formal Methods and Testing.

[11]  Radu Mateescu,et al.  CADP 2011: a toolbox for the construction and analysis of distributed processes , 2012, International Journal on Software Tools for Technology Transfer.

[12]  Rachida Dssouli,et al.  A test case generation tool for conformance testing of SDL systems , 1999, SDL Forum.

[13]  Junichi Suzuki,et al.  A model transformation framework for domain specific languages: An approach using UML and attribute-oriented programming , 2005 .

[14]  Clifton A. Ericson,et al.  Hazard Analysis Techniques for System Safety: Ericson/Hazard Analysis Techniques for System Safety , 2005 .

[15]  Robert M. Keller,et al.  Formal verification of parallel programs , 1976, CACM.

[16]  Clifton A. Ericson,et al.  Hazard Analysis Techniques for System Safety , 2005 .

[17]  Alexander Pretschner,et al.  Model-Based Testing in Practice , 2005, FM.

[18]  Kwang-Ting Cheng,et al.  Automatic Functional Test Generation Using The Extended Finite State Machine Model , 1993, 30th ACM/IEEE Design Automation Conference.

[19]  Joseph G. D'Ambrosio,et al.  Survey of Software Failsafe Techniques for Safety-Critical Automotive Applications , 2005 .

[20]  Joseph G. D'Ambrosio,et al.  Effective Application of Software Safety Techniques for Automotive Embedded Control Systems , 2005 .

[21]  Ivica Crnkovic,et al.  Models transformation between UML and a Domain Specific Language , 2008 .

[22]  Gary McGraw,et al.  Software fault injection: inoculating programs against errors , 1997 .

[23]  Beatrice Gralton,et al.  Washington DC - USA , 2008 .

[24]  YoungJoon Byun,et al.  Design Patterns of Communicating Extended Finite State Machines in SDL , 2001 .

[25]  Ben Swarup Medikonda P. Seetha Ramaiah Anu A. Gokhale,et al.  FMEA and Fault Tree based Software Safety Analysis of a Railroad Crossing Critical System , 2011 .

[26]  Enrico Giunchiglia,et al.  Automatic Test Generation for Coverage Analysis of ERTMS Software , 2009, 2009 International Conference on Software Testing Verification and Validation.

[27]  Doo-Hwan Bae,et al.  Bridging the Gap between Fault Trees and UML State Machine Diagrams for Safety Analysis , 2010, 2010 Asia Pacific Software Engineering Conference.

[28]  Robert Eschbach,et al.  Risk-Based Testing of Safety-Critical Embedded Systems Driven by Fault Tree Analysis , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[29]  Bernhard Kaiser.  A Fault-Tree Semantics to model Software-Controlled Systems , 2003, Softwaretechnik-Trends.

[30]  Francesco Flammini,et al.  A MULTIFORMALISM MODULAR APPROACH TO ERTMS/ETCS FAILURE MODELING , 2014 .

[31]  Anneliese Amschler Andrews,et al.  Fail-Safe Testing of Safety-Critical Systems , 2014, 2014 23rd Australian Software Engineering Conference.

[32]  Luigi Portinale,et al.  Radyban: A tool for reliability analysis of dynamic fault trees through conversion into dynamic Bayesian networks , 2008, Reliab. Eng. Syst. Saf..

[33]  Thomas Bauer,et al.  Automated Risk-Based Testing by Integrating Safety Analysis Information into System Behavior Models , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops.

[34]  Frank Ortmeier,et al.  FORMAL FAILURE MODELS , 2007 .

[35]  Alexandre Petrenko,et al.  Test Generation for CEFSM Combining Specification and Fault Coverage , 2002, TestCom.

[36]  A.C. Tribble,et al.  Software intensive systems safety analysis , 2004, IEEE Aerospace and Electronic Systems Magazine.

[37]  Beverly A. Sanders,et al.  A pattern language for communication pro - tocols , 2002 .

[38]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[39]  Dianxiang Xu,et al.  Integrating Safety Analysis With Functional Modeling , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[40]  Bernhard Kaiser,et al.  State/event fault trees - A safety analysis model for software-controlled systems , 2007, Reliab. Eng. Syst. Saf..

[41]  W. Eric Wong,et al.  Automatic test generation from communicating extended finite state machine (CEFSM)-based models , 2002, Proceedings Fifth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing. ISIRC 2002.

[42]  Miguel Felder,et al.  A Systematic Approach to Generate Test Cases based on Faults , 2003 .

[43]  Mohamed Ghazel,et al.  Formalizing a subset of ERTMS/ETCS specifications for verification purposes , 2014 .

[44]  P. Savage,et al.  Automated test methodology for Operational Flight Programs , 1997, 1997 IEEE Aerospace Conference.

[45]  S. Rahman.  Reliability Engineering and System Safety , 2011 .

[46]  Beverly A. Sanders,et al.  Pattern-based design and validation of communication protocols , 2003 .

[47]  Rachida Dssouli,et al.  A test case generation approach for conformance testing of SDL systems , 2001, Comput. Commun..

[48]  Yanxiang He,et al.  Formal fault tree construction and system safety analysis , 2004, IASTED Conf. on Software Engineering.

[49]  Stefano Marrone,et al.  Using repairable fault trees for the evaluation of design choices for critical repairable systems , 2005, Ninth IEEE International Symposium on High-Assurance Systems Engineering (HASE'05).

[50]  Olivier Ponsini,et al.  Verification of an industrial SystemC/TLM model using LOTOS and CADP , 2009, 2009 7th IEEE/ACM International Conference on Formal Methods and Models for Co-Design.

[51]  Bernhard Rumpe,et al.  Model-driven Development of Complex Software : A Research Roadmap , 2007 .

[52]  Zoltán Pap,et al.  Automatic Test Selection based on CEFSM Specifications , 2002, Acta Cybern..

[53]  Joanne Bechta Dugan,et al.  A discrete-time Bayesian network reliability modeling and analysis framework , 2005, Reliab. Eng. Syst. Saf..

[54]  Ivica Crnkovic,et al.  Using UML for Domain-Specific Component Models , 2009 .

[55]  Nancy G. Leveson,et al.  Analyzing Software Safety , 1983, IEEE Transactions on Software Engineering.

[56]  YoungJoon Byun,et al.  A pattern-based development methodology for communication protocols , 2005, SAC '05.

[57]  Alan Bundy,et al.  Constructing Induction Rules for Deductive Synthesis Proofs , 2006, CLASE.

[58]  Anneliese Andrews,et al.  Testing of safety-critical systems: An aerospace launch application , 2014, 2014 IEEE Aerospace Conference.

[59]  Daniel Brand,et al.  On Communicating Finite-State Machines , 1983, JACM.

[60]  Joseph G. D'Ambrosio,et al.  A Comprehensive Hazard Analysis Technique for Safety-Critical Automotive Systems , 2001 .

[61]  Valeria Vittorini,et al.  Repairable fault tree for the automatic evaluation of repair policies , 2004, International Conference on Dependable Systems and Networks, 2004.

[62]  Luigi Portinale,et al.  Improving the analysis of dependable systems by mapping fault trees into Bayesian networks , 2001, Reliab. Eng. Syst. Saf..

[63]  Hasan Ural,et al.  Automatic Generation of Test Purposes for Testing Distributed Systems , 2003, FATES.

[64]  Makis Stamatelatos,et al.  Fault tree handbook with aerospace applications , 2002 .

[65]  Peter Liggesmeyer,et al.  A New Component Concept for Fault Trees , 2003, SCS.

[66]  B. Kaiser,et al.  Extending the expressive power of fault trees , 2005, Annual Reliability and Maintainability Symposium, 2005. Proceedings..