Towards model-based generation of self-priming and self-checking conformance tests for interactive systems

This paper describes a model-based approach to generate conformance tests for interactive applications. Our method builds on existing work to address generation of: (1) small yet effective set of test frames for testing individual operations, (2) a Set up sequence that brings the system under test in an appropriate state for a test frame (self-priming), (3) a Verification sequence for expected output and state changes (self-checking), and, (4) negative test cases in the presence of exceptions. Our method exploits a novel mutation scheme applied to operations specified as pre-, and postconditions on parameters and state variables; a set of novel abstraction techniques which result in a compact finite state automaton; and search techniques to automatically generate the set up and verification sequences. We illustrate our method with a simple ATM application.

[1]  Paul Ammann,et al.  Using model checking to generate tests from specifications , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).

[2]  Mladen A. Vouk,et al.  Automatic test generation for predicates , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[3]  Alan Hartman,et al.  Using a model-based test generator to test for standard conformance , 2002, IBM Syst. J..

[4]  Marc J. Balcer,et al.  The category-partition method for specifying and generating fuctional tests , 1988, CACM.

[5]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[6]  Robert M. Hierons,et al.  Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[7]  Amit M. Paradkar SALT-an integrated environment to automate generation of function tests for APIs , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[8]  Adele E. Howe,et al.  Test Case Generation as an AI Planning Problem , 2004, Automated Software Engineering.

[9]  Ibrahim Khalil,et al.  Automated Construction of Software Behavior Models , 1999 .

[10]  Ana R. Cavalli,et al.  An automatic and optimized test generation technique applying to TCP/IP protocol , 1999, 14th IEEE International Conference on Automated Software Engineering.

[11]  Patrick Cousot,et al.  Systematic design of program analysis frameworks , 1979, POPL.

[12]  Mary Lou Soffa,et al.  Using a goal-driven approach to generate test cases for GUIs , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[13]  Angelo Gargantini,et al.  Using model checking to generate tests from requirements specifications , 1999, ESEC/FSE-7.

[14]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[15]  James A. Whittaker,et al.  A Markov Chain Model for Statistical Software Testing , 1994, IEEE Trans. Software Eng..

[16]  Thomas J. Ostrand,et al.  Automatic generation of test scripts from formal test specifications , 1989 .

[17]  Peter Fröhlich,et al.  Automated Test Case Generation from Dynamic Models , 2000, ECOOP.

[18]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.