Generating online test oracles from temporal logic specifications

This paper is concerned with the problem of checking, by means of testing, that a software component satisfies a specification of temporal safety properties. Checking that an actual observed behavior conforms to the specification is performed by a test oracle, which can be either a human tester or a software module. We present a technique for automatically generating test oracles from specifications of temporal safety properties in a metric temporal logic. The logic can express quantitative timing properties, and can also express properties of data values by means of a quantification construct. The generated oracle works online in the sense that checking is performed simultaneously with observation. The technique has been implemented and used in case studies at Volvo Technical Development Corporation .

[1]  Ioannis Parissis,et al.  Testing synchronous critical software , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[2]  Zohar Manna,et al.  A Decision Algorithm for Full Propositional Temporal Logic , 1993, CAV.

[3]  Leslie Lamport,et al.  Proving the Correctness of Multiprocess Programs , 1977, IEEE Transactions on Software Engineering.

[4]  Paul Gastin,et al.  Fast LTL to Büchi Automata Translation , 2001, CAV.

[5]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[6]  Sandro Morasca,et al.  Generating test cases for real-time systems from logic specifications , 1995, TOCS.

[7]  Kenneth L. McMillan,et al.  Symbolic model checking , 1992 .

[8]  George S. Avrunin,et al.  Patterns in property specifications for finite-state verification , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[9]  Fausto Giunchiglia,et al.  Improved Automata Generation for Linear Temporal Logic , 1999, CAV.

[10]  Ioannis Parissis,et al.  Specification-based testing of synchronous software , 1996, SIGSOFT '96.

[11]  Kousha Etessami,et al.  Optimizing Büchi Automata , 2000, CONCUR.

[12]  Grigore Rosu,et al.  Synthesizing Monitors for Safety Properties , 2002, TACAS.

[13]  Doron Drusinsky,et al.  The Temporal Rover and the ATG Rover , 2000, SPIN.

[14]  Zohar Manna,et al.  Temporal Verification of Reactive Systems , 1995, Springer New York.

[15]  Thomas A. Henzinger,et al.  A really temporal logic , 1994, JACM.

[16]  Miguel Felder,et al.  Validating real-time systems by history-checking TRIO specifications , 1992, ICSE '92.

[17]  Pierre Wolper,et al.  An Automata-Theoretic Approach to Automatic Program Verification (Preliminary Report) , 1986, LICS.

[18]  Eugene Miya,et al.  On "Software engineering" , 1985, SOEN.

[19]  Nicolas Halbwachs,et al.  Automatic testing of reactive systems , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[20]  Zohar Manna,et al.  Temporal verification of reactive systems - safety , 1995 .

[21]  Michael Fisher,et al.  A Normal Form for Temporal Logics and its Applications in Theorem-Proving and Execution , 1997, J. Log. Comput..

[22]  David Lorge Parnas,et al.  Using Test Oracles Generated from Program Documentation , 1998, IEEE Trans. Software Eng..

[23]  Zohar Manna,et al.  The Temporal Logic of Reactive and Concurrent Systems , 1991, Springer New York.

[24]  Amir Pnueli,et al.  Algorithmic Verification of Linear Temporal Logic Specifications , 1998, ICALP.

[25]  Ilan Beer,et al.  FoCs: Automatic Generation of Simulation Checkers from Formal Specifications , 2000, CAV.

[26]  Fabio Somenzi,et al.  Efficient Büchi Automata from LTL Formulae , 2000, CAV.