Verification and testing of safety-critical airborne systems: A model-based methodology

In this paper, we address the issues of safety-critical software verification and testing that are key requirements for achieving DO-178C and DO331 regulatory compliance for airborne systems. Formal verification and testing are considered two different activities within airborne standards and they belong to two different levels in the avionics software development cycle. The objective is to integrate model-based verification and model-based testing within a single framework and to capture the benefits of their cross-fertilization. This is achieved by proposing a new methodology for the verification and testing of parallel communicating agents based on formal models. In this work, properties are extracted from requirements and formally verified at the design level, while the verified properties are propagated to the implementation level and checked via testing. The contributions of this paper are a methodology that integrates verification and testing, formal verification of some safety critical software properties, and a testing method for Modified Condition/Decision Coverage (MC/DC). The results of formal verification and testing can be used as evidence for avionics software certification.

[1]  Felix Kossak.  Landing Gear System: An ASM-Based Solution for the ABZ Case Study , 2014, ABZ.

[2]  Paolo Arcaini,et al.  Offline Model-Based Testing and Runtime Monitoring of the Sensor Voting Module , 2014, ABZ.

[3]  Jamal Bentahar,et al.  Conditional Commitments , 2014, ACM Trans. Softw. Eng. Methodol..

[4]  Thierry Jéron,et al.  Test Generation Derived from Model-Checking , 1999, CAV.

[5]  Edmund M. Clarke,et al.  Formal Methods: State of the Art and Future Directions Working Group Members , 1996 .

[6]  Rachida Dssouli,et al.  Automatic executable test case generation for extended finite state machine protocols , 1997 .

[7]  Alessio Lomuscio,et al.  MCMAS: A Model Checker for the Verification of Multi-Agent Systems , 2009, CAV.

[8]  Gordon Fraser,et al.  Testing with model checkers: a survey , 2009, Softw. Test. Verification Reliab..

[9]  Stephan Merz,et al.  Model Checking , 2000 .

[10]  Bernhard K. Aichernig,et al.  From Faults Via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems , 2006, FASE.

[11]  Rachida Dssouli,et al.  A test case generation approach for conformance testing of SDL systems , 2001, Comput. Commun..

[12]  Dominique Méry,et al.  Modeling an Aircraft Landing System in Event-B , 2014, ABZ.

[13]  Paul E. Black.  Modeling and marshaling: making tests from model checker counterexamples , 2000, 19th DASC. 19th Digital Avionics Systems Conference. Proceedings (Cat. No.00CH37126).

[14]  Gregory Zoughbi,et al.  Modeling safety and airworthiness (RTCA DO-178B) information: conceptual model and UML profile , 2011, Software & Systems Modeling.

[15]  Jean-Raymond Abrial,et al.  Aircraft landing gear system: approaches with Event-B to the modeling of an industrial system , 2015, International Journal on Software Tools for Technology Transfer.

[16]  Alexander Pretschner.  Compositional Generation of MC/DC Integration Test Suites , 2003, Electron. Notes Theor. Comput. Sci..

[17]  Amel Mammar,et al.  Modeling a landing gear system in Event-B , 2015, International Journal on Software Tools for Technology Transfer.

[18]  Jamal Bentahar,et al.  Formal Specification and Automatic Verification of Conditional Commitments , 2015, IEEE Intelligent Systems.

[19]  Ferhat Khendek,et al.  Compositional Testing of Communication Systems , 2006, TestCom.

[20]  Jianping Wu,et al.  Reachability Graph Based Hierarchical Test Generation for Network Protocols Modeled as Parallel Finite State Machines , 2013, 2013 22nd International Conference on Computer Communication and Networks (ICCCN).

[21]  Paolo Arcaini,et al.  Modeling and Analyzing Using ASMs: The Landing Gear System Case Study , 2014, ABZ.

[22]  Silvano Dal-Zilio,et al.  Model-Checking Real-Time Properties of an Aircraft Landing Gear System Using Fiacre , 2014, ABZ.

[23]  Jan Peleska,et al.  Test automation of safety-critical reactive systems , 1997 .

[24]  Michael Leuschel,et al.  Validation of the ABZ landing gear system using ProB , 2015, International Journal on Software Tools for Technology Transfer.

[25]  Edmund M. Clarke,et al.  Efficient generation of counterexamples and witnesses in symbolic model checking , 1995, DAC '95.

[26]  Yannick Moy,et al.  Testing or Formal Verification: DO-178C Alternatives and Industrial Experience , 2013, IEEE Software.

[27]  Robert M. Hierons,et al.  Generating Feasible Transition Paths for Testing from an Extended Finite State Machine (EFSM) , 2009, 2009 International Conference on Software Testing Verification and Validation.

[28]  Jianping Wu,et al.  Modeling and Testing of Network Protocols with Parallel State Machines , 2015, IEICE Trans. Inf. Syst..

[29]  Jan Peleska,et al.  Industrial-Strength Model-Based Testing - State of the Art and Current Challenges , 2013, MBT.

[30]  Frédéric Boniol,et al.  The Landing Gear System Case Study , 2014, ABZ.

[31]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[32]  Gordon Fraser,et al.  Issues in using model checkers for test case generation , 2009, J. Syst. Softw..

[33]  John M. Rushby,et al.  New challenges in certification for aircraft software , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[34]  Helmut Veith,et al.  Counterexamples Revisited: Principles, Algorithms, Applications , 2003, Verification: Theory and Practice.

[35]  Michael J. Butler,et al.  Co-simulation Environment for Rodin: Landing Gear Case Study , 2014, ABZ.

[36]  Ismail Berrada,et al.  Testing Communicating Systems: a Model, a Methodology, and a Tool , 2005, TestCom.

[37]  Rachida Dssouli,et al.  A Guided Incremental Test Case Generation Procedure for Conformance Testing for CEFSM Specified Protocols , 1998, IWTCS.

[38]  Richard Banach.  The Landing Gear Case Study in Hybrid Event-B , 2014, ABZ.

[39]  Philippe Dhaussy,et al.  Context-Aware Verification of a Landing Gear System , 2014, ABZ.

[40]  Bruno Legeard,et al.  A taxonomy of model-based testing , 2006 .