Model-Based Testing for Avionic Systems Proven Benefits and Further Challenges

In this article, we report on the transition of model-based testing (MBT) from a widely discussed research discipline to an accepted technology that is currently becoming state of the art in industry; in particular, in the field of safety-critical systems testing. It is reviewed how focal points of MBT-related research in the past have found their way into today’s commercial MBT products. We describe the benefits of MBT that are – from our experience – most appreciated by practitioners. Moreover, some interesting open challenges are described, and potential future solutions are presented. The material presented in this paper is based on our practical experience with recent MBT campaigns performed for Airbus in Germany.

[1]  Jörg Brauer,et al.  Efficient and Trustworthy Tool Qualification for Model-Based Testing Tools , 2012, ICTSS.

[2]  Jan Peleska,et al.  Industrial-Strength Model-Based Testing - State of the Art and Current Challenges , 2013, MBT.

[3]  Michael Felderer,et al.  A Case Study on the Efficiency of Model-Based Testing at the European Space Agency , 2015, 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST).

[4]  Jan Peleska.  Model-based avionic systems testing for the airbus family , 2018, 2018 IEEE 23rd European Test Symposium (ETS).

[5]  M. P. Vasilevskii.  Failure diagnosis of automata , 1973 .

[6]  A. Prasad Sistla,et al.  Safety, liveness and fairness in temporal logic , 1994, Formal Aspects of Computing.

[7]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[8]  Michael Felderer,et al.  Estimating the Cost and Benefit of Model-Based Testing: A Decision Support Procedure for the Application of Model-Based Testing in Industry , 2015, 2015 41st Euromicro Conference on Software Engineering and Advanced Applications.

[9]  Olivier Casse,et al.  SysML: Object Management Group (OMG) Systems Modeling Language , 2018 .

[10]  Ina Schieferdecker,et al.  Test Models and Coverage Criteria for Automatic Model-Based Test Generation with UML State Machines , 2010 .

[11]  Jan Peleska,et al.  Automated Test Case Generation with SMT-Solving and Abstract Interpretation , 2011, NASA Formal Methods.

[12]  Jörg Brauer,et al.  Model-Based Testing for Avionics Systems , 2018, FM.

[13]  Yang Liu,et al.  A formal model for the SPARCv8 ISA and a proof of non-interference for the LEON3 processor , 2016, Arch. Formal Proofs.

[14]  Jan Peleska,et al.  A Novel Approach to HW/SW Integration Testing of Route-Based Interlocking System Controllers , 2016, RSSRail.

[15]  Alexandre Petrenko,et al.  Model-based testing of software and systems: recent advances and challenges , 2012, International Journal on Software Tools for Technology Transfer.

[16]  Kim G. Larsen,et al.  Testing Real-Time Systems Using UPPAAL , 2008, Formal Methods and Testing.

[17]  Jan Peleska,et al.  Experimental evaluation of a novel equivalence class partition testing strategy , 2017, Software & Systems Modeling.

[18]  Alexandre Petrenko,et al.  Checking Experiments for Symbolic Input/Output Finite State Machines , 2016, 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[19]  Augusto Sampaio,et al.  Sound conformance testing for cyber-physical systems: Theory and implementation , 2017, Sci. Comput. Program..

[20]  Guilherme Horta Travassos,et al.  A Picture from the Model-Based Testing Area: Concepts, Techniques, and Challenges , 2010, Adv. Comput..

[21]  Kim G. Larsen,et al.  Testing real-time embedded software using UPPAAL-TRON: an industrial case study , 2005, EMSOFT.

[22]  Alessandra Cavarra,et al.  Data Flow Analysis and Testing of Abstract State Machines , 2008, ABZ.

[23]  César Viho,et al.  Formal Test Automation: The Conference Protocol with TGV/TORX , 2000, TestCom.

[24]  Yu Lei,et al.  Introduction to Combinatorial Testing , 2013 .

[25]  Arne Skou,et al.  Modelling and analysis of a collision avoidance protocol using Spin and Uppaal , 1996, The Spin Verification System.

[26]  Stefania Gnesi,et al.  Model Driven Development and Code Generation: An Automotive Case Study , 2007, SDL Forum.

[27]  Jan Peleska,et al.  Complete model-based equivalence class testing , 2014, International Journal on Software Tools for Technology Transfer.

[28]  Jan Peleska,et al.  Model-Based Testing Strategies and Their (In)dependence on Syntactic Model Representations , 2016, FMICS-AVoCS.

[29]  Reinhard Wilhelm,et al.  Timing Validation of Automotive Software , 2008, ISoLA.

[30]  Viktor Schuppan,et al.  Linear Encodings of Bounded LTL Model Checking , 2006, Log. Methods Comput. Sci..

[31]  Sungwon Kang,et al.  A survey on software product line testing , 2012, SPLC '12.

[32]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[33]  Jan Peleska,et al.  Complete model-based equivalence class testing for nondeterministic systems , 2016, Formal Aspects of Computing.