Minimizing Test Execution Time During Test Generation

In the area of model based testing, major improvements have been made in the generation of conformance tests using a model checker. Unfortunately, the execution of the generated test suites tend to be rather time-consuming. In [1] we presented a method to generate the test suites with the shortest execution time providing the required coverage, but this method can only be applied to small models due to memory-consumption. Here we show how to generate test suites for a number of different test quality criteria like coverage criteria, UIOs, mutant testing. Moreover, we present heuristics to significantly reduce test execution time that are as efficient as a naive testsuite generation. Our optimization combines min-set-cover-algorithms and search strategies, which we use to enlengthen generated test cases by promising additional coverages. We compare several heuristics and present a case study where we could achieve a reduction of the test execution time to less than 10%.

[1]  Pierre G. Paulin,et al.  Force-directed scheduling for the behavioral synthesis of ASICs , 1989, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[2]  George Mason,et al.  Procedures for Reducing the Size of Coverage-based Test Sets , 1995 .

[3]  Mats Per Erik Heimdahl,et al.  Test-suite reduction for model based tests: effects on test quality and implications for testing , 2004, Proceedings. 19th International Conference on Automated Software Engineering, 2004..

[4]  Mats Per Erik Heimdahl,et al.  Specification test coverage adequacy criteria = specification test generation inadequacy criteria , 2004, Eighth IEEE International Symposium on High Assurance Systems Engineering, 2004. Proceedings..

[5]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[6]  Peter Liggesmeyer,et al.  Generating optimal distinguishing sequences with a model checker , 2005, A-MOST.

[7]  Anna Philippou,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2018, Lecture Notes in Computer Science.

[8]  Sungdeok Cha Automatic Test Generation from Statecharts Using Model Checking , 2001 .

[9]  Michaela Huhn,et al.  Vooduu: Verification of Object-Oriented Designs Using UPPAAL , 2004, TACAS.

[10]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[11]  W. Eric Wong,et al.  Mutation Testing Applied to Validate SDL Specifications , 2004, TestCom.

[12]  Michaela Huhn,et al.  Generation of Optimized Testsuites for UML Statecharts with Time , 2004, TestCom.

[13]  Wang Yi,et al.  Uppaal in a nutshell , 1997, International Journal on Software Tools for Technology Transfer.

[14]  Loe M. G. Feijs,et al.  Test Generation for Intelligent Networks Using Model Checking , 1997, TACAS.

[15]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[16]  Sanjai Rayadurgam,et al.  Coverage based test-case generation using model checkers , 2001, Proceedings. Eighth Annual IEEE International Conference and Workshop On the Engineering of Computer-Based Systems-ECBS 2001.

[17]  A. Pretschner Classical search strategies for test case generation with Constraint Logic Programming , 2001 .