Testing, Optimizaton, and Games

We discuss algorithmic problems arising in the testing of reactive systems, i.e. systems that interact with their environment. The goal is to design test sequences so that we can deduce desired information about the given system under test, such as whether it conforms to a given specification model, or whether it satisfies given requirement properties. Test generation can be approached from different points of view – as an optimization problem of minimizing cost and maximizing the effectiveness of the tests; as a game between tester and system under test; or as a learning problem. We touch on some of these aspects and related algorithmic questions.

[1]  David Lee,et al.  Testing Finite State Machines: Fault Detection , 1995, J. Comput. Syst. Sci..

[2]  Ronald L. Rivest,et al.  Inference of finite automata using homing sequences , 1989, STOC '89.

[3]  John H. Reif,et al.  The Complexity of Two-Player Games of Incomplete Information , 1984, J. Comput. Syst. Sci..

[4]  L. Apfelbaum,et al.  Automated functional test generation , 1995, Conference Record AUTOTESTCON '95. 'Systems Readiness: Test Technology for the 21st Century'.

[5]  Alfred V. Aho,et al.  An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours , 1991, IEEE Trans. Commun..

[6]  Ivar Jacobson,et al.  The Unified Modeling Language User Guide , 1998, J. Database Manag..

[7]  Alex Groce,et al.  Adaptive Model Checking , 2002, Log. J. IGPL.

[8]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[9]  Insup Lee,et al.  A Temporal Logic Based Theory of Test Coverage and Generation , 2002, TACAS.

[10]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[11]  Edward F. Moore,et al.  Gedanken-Experiments on Sequential Machines , 1956 .

[12]  Alan M. Frieze,et al.  On the worst-case performance of some algorithms for the asymmetric traveling salesman problem , 1982, Networks.

[13]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[14]  David Lee,et al.  Testing Finite-State Machines: State Identification and Verification , 1994, IEEE Trans. Computers.

[15]  Rajeev Alur,et al.  Distinguishing tests for nondeterministic and probabilistic machines , 1995, STOC '95.

[16]  Jack Edmonds,et al.  Matching, Euler tours and the Chinese postman , 1973, Math. Program..

[17]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[18]  Jan Tretmans,et al.  Testing Transition Systems: An Annotated Bibliography , 2000, MOVEP.

[19]  M. P. Vasilevskii.  Failure diagnosis of automata , 1973 .

[20]  Kousha Etessami,et al.  From Rule-based to Automata-based Testing , 2000, FORTE.

[21]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[22]  Alan Hartman,et al.  Projected state machine coverage for software testing , 2002, ISSTA '02.

[23]  Rajeev Alur,et al.  Model checking of hierarchical state machines , 2001, SIGSOFT '98/FSE-6.

[24]  Mihalis Yannakakis,et al.  Black Box Checking , 1999, FORTE.

[25]  Mihalis Yannakakis,et al.  Optimization problems from feature testing of communication protocols , 1996, Proceedings of 1996 International Conference on Network Protocols (ICNP-96).

[26]  David Lee,et al.  Online minimization of transition systems (extended abstract) , 1992, STOC '92.

[27]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[28]  Alexandre Petrenko,et al.  Fault Model-Driven Test Derivation from Finite State Models: Annotated Bibliography , 2000, MOVEP.

[29]  F. C. Hennine.  Fault detecting experiments for sequential circuits , 1964, SWCT 1964.

[30]  Gerard J. Holzmann,et al.  Design tools for requirements engineering , 1997, Bell Labs Technical Journal.