Combining Model Refinement and Test Generation for Conformance Testing of the IEEE PHD Protocol Using Abstract State Machines

In this paper we propose a new approach to conformance testing based on Abstract State Machine (ASM) model refinement. It consists in generating test sequences from ASM models and checking the conformance between code and models in multiple iterations. This process is applied at different models, starting from the more abstract model to the one that is very close to the code. The process consists of the following steps: (1) model the system as an Abstract State Machine, (2) generate test sequences based on the ASM model, (3) compute the code coverage using generated tests, (4) if the coverage is low refine the Abstract State Machine and return to step 2. We have applied the proposed approach to Antidote, an open-source implementation of IEEE 11073-20601 Personal Health Device (PHD) protocol which allows personal healthcare devices to exchange data with other devices such as small computers and smartphones.

[1]  Gregor von Bochmann,et al.  A Test Design Methodology for Protocol Testing , 1987, IEEE Transactions on Software Engineering.

[2]  Kyu-Chul Lee,et al.  ISO/IEEE 11073 PHD message generation toolkit to standardize healthcare device , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[3]  Christophe Gaston,et al.  Off-Line Test Case Generation for Timed Symbolic Model-Based Conformance Testing , 2012, ICTSS.

[4]  Angelo Gargantini,et al.  Generation of C++ Unit Tests from Abstract State Machines Specifications , 2018, 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[5]  Paolo Arcaini,et al.  Automatic Refinement of ASM Abstract Test Cases , 2019, 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[6]  Myla Archer,et al.  Model-based construction and verification of critical systems using composition and partial refinement , 2010, Formal Methods Syst. Des..

[7]  Sandra Martinez,et al.  Medical Device Communication : A Standards-based Conformance Testing Approach , 2008 .

[8]  Angelo Gargantini,et al.  Validation of Transformation from Abstract State Machine Models to C++ Code , 2018, ICTSS.

[9]  Bernhard K. Aichernig,et al.  Model-based mutation testing via symbolic refinement checking , 2015, Sci. Comput. Program..

[10]  Paolo Arcaini,et al.  Repairing Timed Automata Clock Guards through Abstraction and Testing , 2019, TAP@FM.

[11]  Ana Maria Ambrosio,et al.  FSM-Based Test Case Generation Methods Applied to Test the Communication Software on Board the ITASAT University Satellite: A Case Study , 2014 .

[12]  Jeff Yu Lei,et al.  A general conformance testing framework for IEEE 11073 PHD's communication model , 2013, PETRA '13.

[13]  Angelo Gargantini,et al.  Asm2C++: A Tool for Code Generation from Abstract State Machines to Arduino , 2017, NFM.

[14]  Ana R. Cavalli,et al.  FSM-based conformance testing methods: A survey annotated with experimental evaluation , 2010, Inf. Softw. Technol..

[15]  Mariam Lahami,et al.  A model-based approach to combine conformance and load tests: an eHealth case study , 2018 .

[16]  Yang Zhao,et al.  Formal specification and verification of a coordination protocol for an automated air traffic control system , 2012, Sci. Comput. Program..

[17]  Frits W. Vaandrager,et al.  Testing timed automata , 1997, Theor. Comput. Sci..

[18]  Paolo Arcaini,et al.  A model‐driven process for engineering a toolset for a formal method , 2011, Softw. Pract. Exp..

[19]  Alessandro Cimatti,et al.  Tightening the contract refinements of a system architecture , 2018, Formal Methods Syst. Des..

[20]  Achim D. Brucker,et al.  Model-Based Firewall Conformance Testing , 2008, TestCom/FATES.

[21]  Paolo Arcaini,et al.  SMT-Based Automatic Proof of ASM Model Refinement , 2016, SEFM.

[22]  Hadi Hemmati,et al.  How Effective Are Code Coverage Criteria? , 2015, 2015 IEEE International Conference on Software Quality, Reliability and Security.

[23]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[24]  Sébastien Salva,et al.  A Model-Based Testing Approach Combining Passive Conformance Testing and Runtime Verification: Application to Web Service Compositions Deployed in Clouds , 2013, SERA.

[25]  Teruo Higashino,et al.  A conformance testing method for communication protocols modeled as concurrent DFSMs. Treatment of non-observable non-determinism , 2001, Proceedings 15th International Conference on Information Networking.

[26]  Asuman Dogac,et al.  An Interoperability Test Framework for HL7-Based Systems , 2009, IEEE Transactions on Information Technology in Biomedicine.

[27]  Florian Lorber,et al.  Time for Mutants - Model-Based Mutation Testing with Timed Automata , 2013, TAP@STAF.

[28]  Egon Börger,et al.  Abstract State Machines. A Method for High-Level System Design and Analysis , 2003 .

[29]  Radu Mateescu,et al.  TESTOR: A Modular Tool for On-the-Fly Conformance Test Case Generation , 2018, TACAS.

[30]  Bernhard K. Aichernig,et al.  Protocol Conformance Testing a SIP Registrar: an Industrial Application of Formal Methods , 2007, Fifth IEEE International Conference on Software Engineering and Formal Methods (SEFM 2007).

[31]  Mark Skall,et al.  Conformance Testing and Interoperability: A Case Study in Healthcare Data Exchange , 2008, Software Engineering Research and Practice.

[32]  Angelo Gargantini,et al.  ASM-Based Testing: Coverage Criteria and Automatic Test Sequence , 2001, J. Univers. Comput. Sci..