An Expressive and Implementable Formal Framework for Testing Real-Time Systems

We propose a new framework for black-box conformance testing of real-time systems, based on the model of timed automata. The framework is expressive: it can fully handle partially-observable, non-deterministic timed automata. It also allows the user to define, through appropriate modeling, assumptions on the environment of the system under test (SUT) as well as on the interface between the tester and the SUT. The framework is implementable: tests can be implemented as finite-state machines accessing a finite-precision digital clock. We propose, for this framework, a set of test-generation algorithms with respect to different coverage criteria. We have implemented these algorithms in a prototype tool called TTG. Experimental results obtained by applying TTG on the Bounded Retransmission Protocol show that only a few tests suffice to cover thousands of reachable symbolic states in the specification.

[1]  Stavros Tripakis,et al.  Fault Diagnosis for Timed Automata , 2002, FTRTFT.

[2]  Jan Tretmans,et al.  Testing Concurrent Systems: A Formal Approach , 1999, CONCUR.

[3]  Joost-Pieter Katoen,et al.  A probabilistic extension of UML statecharts: Specification and Verification. , 2002 .

[4]  Ferhat Khendek,et al.  Timed test cases generation based on state characterization technique , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[5]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[6]  Stavros Tripakis,et al.  Folk Theorems on the Determinization and Minimization of Timed Automata , 2003, FORMATS.

[7]  Duncan Clarke,et al.  STG: A Symbolic Test Generation Tool , 2002, TACAS.

[8]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[9]  Stavros Tripakis,et al.  Testing Conformance of Real-Time Applications by Automatic Generation of Observers , 2005, Electron. Notes Theor. Comput. Sci..

[10]  Thomas A. Henzinger,et al.  A Determinizable Class of Timed Automata , 1994, CAV.

[11]  David L. Dill,et al.  Timing Assumptions and Verification of Finite-State Concurrent Systems , 1989, Automatic Verification Methods for Finite State Systems.

[12]  Paul Gastin,et al.  Characterization of the Expressive Power of Silent Transitions in Timed Automata , 1998, Fundam. Informaticae.

[13]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[14]  Joseph Sifakis,et al.  Modeling Urgency in Timed Systems , 1997, COMPOS.

[15]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[16]  Thomas A. Henzinger,et al.  What Good Are Digital Clocks? , 1992, ICALP.

[17]  Brian Nielsen,et al.  Automated test generation from timed automata , 2001, International Journal on Software Tools for Technology Transfer.

[18]  J. Taylor,et al.  Switching and finite automata theory, 2nd ed. , 1980, Proceedings of the IEEE.

[19]  Brian Nielsen,et al.  Automated Test Generation from Timed Automata , 2001, TACAS.

[20]  I. Lee,et al.  Automatic generation of tests for timing constraints from requirements , 1997, Proceedings Third International Workshop on Object-Oriented Real-Time Dependable Systems.

[21]  Joseph Sifakis,et al.  Automatic Verification Methods for Finite State Systems , 1989, Lecture Notes in Computer Science.

[22]  Jan Peleska.  Formal methods for test automation-hard real-time testing of con-trollers for the airbus aircraft fa , 2002 .

[23]  Manfred Broy,et al.  Perspectives of System Informatics , 2001, Lecture Notes in Computer Science.

[24]  Stavros Tripakis,et al.  Conformance testing for real-time systems , 2004, SPIN.

[25]  Brian Nielsen,et al.  Formal Approaches to Software Testing, 4th International Workshop, FATES 2004, Linz, Austria, September 21, 2004, Revised Selected Papers , 2005, FATES.

[26]  Stavros Tripakis,et al.  The Tool KRONOS , 1996, Hybrid Systems.

[27]  Frits W. Vaandrager,et al.  Testing timed automata , 1997, Theor. Comput. Sci..

[28]  Bernard Berthomieu,et al.  An Enumerative Approach for Analyzing Time Petri Nets , 1983, IFIP Congress.

[29]  Nicolae Goga,et al.  Formal Test Automation: A Simple Experiment , 1999, IWTCS.

[30]  Kim G. Larsen,et al.  Time-Optimal Real-Time Test Case Generation Using Uppaal , 2003, FATES.

[31]  Jan Tretmans,et al.  Testing Transition Systems: An Annotated Bibliography , 2000, MOVEP.

[32]  Alexandre Petrenko,et al.  Integration of Functional and Timed Testing of Real-Time and Concurrent Systems , 2003, Ershov Memorial Conference.

[33]  Marius Bozga,et al.  Experiment on Verification of a Planetary Rover Controller , 2004 .

[34]  Rachel Cardell-Oliver.  Conformance Tests for Real-Time Systems with Timed Automata Specifications , 2000, Formal Aspects of Computing.

[35]  Iulian Ober,et al.  SDL for Real-Time: What is Missing? , 2000, SAM.

[36]  Rachel Cardell-Oliver,et al.  A Practical and Complete Algorithm for Testing Real-Time Systems , 1998, FTRTFT.

[37]  Alexandre Petrenko,et al.  Confirming configurations in EFSM testing , 2004, IEEE Transactions on Software Engineering.

[38]  Edward A. Feigenbaum,et al.  Switching and Finite Automata Theory: Computer Science Series , 1990 .

[39]  Thierry Jéron,et al.  Using On-The-Fly Verification Techniques for the Generation of test Suites , 1996, CAV.

[40]  C. A. Petri,et al.  Concurrency Theory , 1986, Advances in Petri Nets.

[41]  Miguel Felder,et al.  Testing Timing Behavior of Real-Time Software , 1997 .

[42]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[43]  Marius Bozga,et al.  IF: A Validation Environment for Timed Asynchronous Systems , 2000, CAV.

[44]  Rachel Cardell-Oliver.  Conformance test experiments for distributed real-time systems , 2002, ISSTA '02.

[45]  Teruo Higashino,et al.  Generating Test Cases for a Timed I/O Automaton Model , 1999, IWTCS.

[46]  Stavros Tripakis,et al.  Analysis of Timed Systems Using Time-Abstracting Bisimulations , 2001, Formal Methods Syst. Des..

[47]  David Lee,et al.  Testing finite state machines , 1991, STOC '91.

[48]  Thierry Jéron,et al.  Test Cases Generation for Nondeterministic Real-Time Systems , 2003, FATES.