Testing Real-Time Systems Using UPPAAL

This chapter presents principles and techniques for modelbased black-box conformance testing of real-time systems using the Uppaal model-checking tool-suite. The basis for testing is given as a network of concurrent timed automata specified by the test engineer. Relativized input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases can be generated offline and later executed, or they can be generated and executed online. For both approaches this chapter discusses how to specify test objectives, derive test sequences, apply these to the system under test, and assign a verdict.

[1]  Kim G. Larsen,et al.  Time-Optimal Real-Time Test Case Generation Using Uppaal , 2003, FATES.

[2]  Manuel Núñez,et al.  Conformance Testing Relations for Timed Systems , 2005, FATES.

[3]  Miguel Felder,et al.  Testing Timing Behavior of Real-Time Software , 1997 .

[4]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[5]  Marius Mikucionis,et al.  T-UPPAAL: online model-based testing of real-time systems , 2004 .

[6]  K. Larsen,et al.  Online Testing of Real-time Systems Using Uppaal , 2004, FATES.

[7]  Stefania Gnesi,et al.  FME 2003: Formal Methods: International Symposium of Formal Methods Europe, Pisa, Italy, September 8-14, 2003. Proceedings , 2003, Lecture Notes in Computer Science.

[8]  Rachel Cardell-Oliver.  Conformance Tests for Real-Time Systems with Timed Automata Specifications , 2000, Formal Aspects of Computing.

[9]  Chris J. Myers,et al.  Automatic Verification of Timed Circuits , 1994, CAV.

[10]  Teruo Higashino,et al.  Generating Test Cases for a Timed I/O Automaton Model , 1999, IWTCS.

[11]  Wang Yi,et al.  UPPAAL Implementation Secrets , 2002, FTRTFT.

[12]  Fabrice Bouquet,et al.  Reification of Executable Test Scripts in Formal Specicifation-Based Test Generation: The Java Card Transaction Mechanism Case Study , 2003, FME.

[13]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[14]  Paul Gastin,et al.  Removing epsilon-Transitions in Timed Automata , 1997, STACS.

[15]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[16]  Insup Lee,et al.  Data flow testing as model checking , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[17]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[18]  Boudewijn R. Haverkort,et al.  Formal Methods: Applications and Technology, 11th International Workshop, FMICS 2006 and 5th International Workshop PDMC 2006, Bonn, Germany, August 26-27, and August 31, 2006, Revised Selected Papers , 2007, FMICS/PDMC.

[19]  Insup Lee,et al.  A Temporal Logic Based Theory of Test Coverage and Generation , 2002, TACAS.

[20]  Jan Tretmans,et al.  On-the-fly conformance testing using SPIN , 2000, International Journal on Software Tools for Technology Transfer.

[21]  Brian Nielsen,et al.  Formal Approaches to Software Testing, 4th International Workshop, FATES 2004, Linz, Austria, September 21, 2004, Revised Selected Papers , 2005, FATES.

[22]  Conrado Daws,et al.  Verifying ET-LOTOS programmes with KRONOS , 1994, FORTE.

[23]  Nicolae Goga,et al.  Formal Test Automation: A Simple Experiment , 1999, IWTCS.

[24]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[25]  Jan Tretmans,et al.  Testing Concurrent Systems: A Formal Approach , 1999, CONCUR.

[26]  P. M. Herman,et al.  A Data Flow Analysis Approach to Program Testing , 1976, Aust. Comput. J..

[27]  Thierry Jéron,et al.  Test Generation Derived from Model-Checking , 1999, CAV.

[28]  Thomas A. Henzinger,et al.  What Good Are Digital Clocks? , 1992, ICALP.

[29]  David L. Dill,et al.  Timing Assumptions and Verification of Finite-State Concurrent Systems , 1989, Automatic Verification Methods for Finite State Systems.

[30]  Kim G. Larsen,et al.  Testing real-time embedded software using UPPAAL-TRON: an industrial case study , 2005, EMSOFT.

[31]  Wang Yi,et al.  Diagnostic Model-Checking for Real-Time Systems , 1995, Hybrid Systems.

[32]  C. A. Petri,et al.  Concurrency Theory , 1986, Advances in Petri Nets.

[33]  Sandro Morasca,et al.  Generating test cases for real-time systems from logic specifications , 1995, TOCS.

[34]  A. W. Heerink,et al.  Côte de Resyste in Progress , 2000 .

[35]  Rance Cleaveland,et al.  A theory of testing for real-time , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[36]  Sanjai Rayadurgam,et al.  Auto-generating test sequences using model checkers , 2004 .

[37]  Brian Nielsen,et al.  Automated test generation from timed automata , 2001, International Journal on Software Tools for Technology Transfer.

[38]  Kousha Etessami,et al.  Verifying Probabilistic Procedural Programs , 2004, FSTTCS.

[39]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[40]  Bengt Jonsson,et al.  Automated test generation for industrial Erlang applications , 2003, ERLANG '03.

[41]  Joseph Sifakis,et al.  Automatic Verification Methods for Finite State Systems , 1989, Lecture Notes in Computer Science.

[42]  Alan Hartman,et al.  Projected state machine coverage for software testing , 2002, ISSTA '02.

[43]  Janusz W. Laski,et al.  A Data Flow Oriented Program Testing Strategy , 1983, IEEE Transactions on Software Engineering.

[44]  Stavros Tripakis,et al.  Conformance testing for real-time systems , 2004, SPIN.

[45]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[46]  Brian Nielsen,et al.  Automated Test Generation from Timed Automata , 2001, TACAS.

[47]  Alfred V. Aho,et al.  An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours , 1991, IEEE Trans. Commun..

[48]  Steven P. Miller,et al.  Applicability of modified condition/decision coverage to software testing , 1994, Softw. Eng. J..

[49]  Ed Brinksma,et al.  A Test Generation Framework for quiescent Real-Time Systems , 2004, FATES.

[50]  Thierry Jéron,et al.  An Approach to Symbolic Test Generation , 2000, IFM.

[51]  Hoyt Lougee,et al.  SOFTWARE CONSIDERATIONS IN AIRBORNE SYSTEMS AND EQUIPMENT CERTIFICATION , 2001 .

[52]  Paul D. Amer,et al.  Testing protocols modeled as FSMs with timing parameters , 1999, Comput. Networks.

[53]  Simeon C. Ntafos,et al.  A Comparison of Some Structural Testing Strategies , 1988, IEEE Trans. Software Eng..

[54]  Lori A. Clarke,et al.  A Formal Evaluation of Data Flow Path Selection Criteria , 1989, IEEE Trans. Software Eng..

[55]  Thierry Jéron,et al.  An Experiment in Automatic Generation of Test Suites for Protocols with Verification Technology , 1997, Sci. Comput. Program..

[56]  Jan Tretmans,et al.  Automatic Testing with Formal Methods , 1999 .

[57]  Ferhat Khendek,et al.  Timed test cases generation based on state characterization technique , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[58]  Duncan Clarke,et al.  Testing Real-Time Constraints in a Process Algebraic Setting , 1995, 1995 17th International Conference on Software Engineering.

[59]  Peter Liggesmeyer,et al.  Computer Safety, Reliability, and Security , 2004, Lecture Notes in Computer Science.

[60]  Frits W. Vaandrager,et al.  Testing timed automata , 1997, Theor. Comput. Sci..

[61]  Paul Pettersson,et al.  A test case generation algorithm for real-time systems , 2004 .

[62]  Robin Milner,et al.  On Observing Nondeterminism and Concurrency , 1980, ICALP.

[63]  Paul Pettersson,et al.  Model-Based Testing of a WAP Gateway: , 2006 .

[64]  Joël Ouaknine,et al.  Revisiting digitization, robustness, and decidability for timed automata , 2003, 18th Annual IEEE Symposium of Logic in Computer Science, 2003. Proceedings..

[65]  Marinus J. Plasmeijer,et al.  Gast: Generic Automated Software Testing , 2002, IFL.

[66]  Werner Kluge,et al.  Implementation of Functional Languages , 1996, Lecture Notes in Computer Science.

[67]  Thierry Jéron,et al.  Test Cases Generation for Nondeterministic Real-Time Systems , 2003, FATES.

[68]  Paul Pettersson,et al.  Model-Based Testing of a WAP Gateway: An Industrial Case-Study , 2006, FMICS/PDMC.

[69]  Stavros Tripakis,et al.  Fault Diagnosis for Timed Automata , 2002, FTRTFT.

[70]  Bengt Jonsson,et al.  Specifying and Generating Test Cases Using Observer Automata , 2004, FATES.

[71]  Kim G. Larsen,et al.  As Cheap as Possible: Efficient Cost-Optimal Reachability for Priced Timed Automata , 2001, CAV.

[72]  Rance Cleaveland,et al.  Testing equivalence as a bisimulation equivalence , 1989, Formal Aspects of Computing.

[73]  Gerd Behrmann,et al.  Efficient Guiding Towards Cost-Optimality in UPPAAL , 2001, TACAS.

[74]  Joost-Pieter Katoen,et al.  A probabilistic extension of UML statecharts: Specification and Verification. , 2002 .

[75]  Andreas Podelski,et al.  ACSAR: Software Model Checking with Transfinite Refinement , 2007, SPIN.