Generating efficient test sets with a model checker

It is well-known that counterexamples produced by model checkers can provide a basis for automated generation of test cases. However when this approach is used to meet a coverage criterion, it generally results in very inefficient test sets having many tests and much redundancy. We describe an improved approach that uses model checkers to generate efficient test sets. Furthermore, the generation is itself efficient, and is able to reach deep regions of the statespace. We have prototyped the approach using the model checkers of our SAL system and have applied it to model-based designs developed in Stateflow. In one example, our method achieves complete state and transition coverage in a Stateflow model for the shift scheduler of a 4-speed automatic transmission with a single test case.

[1]  Angelo Gargantini,et al.  Using model checking to generate tests from requirements specifications , 1999, ESEC/FSE-7.

[2]  Jonathan Rees,et al.  Revised3 report on the algorithmic language scheme , 1986, SIGP.

[3]  John R. Callahan,et al.  Automated Software Testing Using Model-Checking , 1996 .

[4]  Natarajan Shankar,et al.  The ICS Decision Procedures for Embedded Deduction , 2004, IJCAR.

[5]  Harald Ruess,et al.  Bounded Model Checking and Induction: From Refutation to Verification (Extended Abstract, Category A) , 2003, CAV.

[6]  Ashish Tiwari,et al.  Sal 2 , 2004, CAV.

[7]  Harald Ruess,et al.  An Experimental Evaluation of Ground Decision Procedures , 2004, CAV.

[8]  Harry Rudin,et al.  Automated Protocol Validation: One Chain of Development , 1978, Comput. Networks.

[9]  Avner Landver,et al.  Coverage-Directed Test Generation Using Symbolic Techniques , 1996, FMCAD.

[10]  S. Ranville,et al.  Winnowing tests: Getting quality coverage from a model checker without quantity , 2001, 20th DASC. 20th Digital Avionics Systems Conference (Cat. No.01CH37219).

[11]  Robert P. Kurshan,et al.  Computer-Aided Verification of Coordinating Processes: The Automata-Theoretic Approach , 2014 .

[12]  Jiang Long,et al.  Smart simulation using collaborative formal and simulation engines , 2000, IEEE/ACM International Conference on Computer Aided Design. ICCAD - 2000. IEEE/ACM Digest of Technical Papers (Cat. No.00CH37140).

[13]  Thomas A. Henzinger,et al.  Generating tests from counterexamples , 2004, Proceedings. 26th International Conference on Software Engineering.

[14]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[15]  Mats Per Erik Heimdahl,et al.  Deviation analysis through model checking , 2002, Proceedings 17th IEEE International Conference on Automated Software Engineering,.

[16]  J Hayhurst Kelly,et al.  A Practical Tutorial on Modified Condition/Decision Coverage , 2001 .

[17]  Harald Ruess,et al.  Bounded Model Checking and Induction: From Refutation to Verification (Extended Abstract, Category A) , 2003, CAV.

[18]  Robert Hieb,et al.  Revised 5 Report on the Algorithmic Language , 1999 .

[19]  Sanjai Rayadurgam,et al.  Auto-generating Test Sequences Using Model Checkers: A Case Study , 2003, FATES.

[20]  R. Kent Dybvig,et al.  Revised5 Report on the Algorithmic Language Scheme , 1986, SIGP.

[21]  Insup Lee,et al.  Data flow testing as model checking , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..