TestEra A Novel Framework for Testing Java Programs y

TestEra is a novel framework for automated specification-based testing of Java programs. Given a formal specification for a method, TestEra uses the method precondition to automatically generate all nonisomorphic test inputs up to a given bound. TestEra executes the method on each test input, and uses the method postcondition as a test oracle to check the correctness of each output. TestEra allows users to give specifications as first-order logic formulae. As an enabling technology, TestEra uses the Alloy toolset, which provides an automatic tool for analyzing first-order logic formulae. We have used TestEra to check several Java programs including a networking architecture, the Alloy-alpha analyzer, and methods from the Java Collection Framework. This article describes the TestEra framework and gives experimental results.

[1]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[2]  Sarfraz Khurshid,et al.  Exploring the design of an intentional naming scheme with an automatic constraint analyzer , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[3]  Ilya Shlyakhter,et al.  Generating effective symmetry-breaking predicates for search problems , 2001, Discrete Applied Mathematics.

[4]  Thomas W. Reps,et al.  Putting static analysis to work for verification: A case study , 2000, ISSTA '00.

[5]  Kent Beck,et al.  Test-infected: programmers love writing tests , 2000 .

[6]  Viktor Kuncak,et al.  Role analysis , 2002, POPL '02.

[7]  Reinhard Wilhelm,et al.  Solving shape-analysis problems in languages with destructive updating , 1998, TOPL.

[8]  Patrice Godefroid,et al.  Model checking for programming languages using VeriSoft , 1997, POPL '97.

[9]  Sarfraz Khurshid,et al.  An analyzable annotation language , 2002, OOPSLA '02.

[10]  Matthew B. Dwyer,et al.  Bandera: extracting finite-state models from Java source code , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[11]  Anders Møller,et al.  The Pointer Assertion Logic Engine , 2000 .

[12]  James C. Corbett,et al.  Bandera: extracting finite-state models from Java source code , 2000, ICSE.

[13]  Daniel Jackson,et al.  Alcoa: the Alloy constraint analyzer , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[14]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[15]  S. L. Gerhart,et al.  Toward a theory of test data selection , 1975, IEEE Transactions on Software Engineering.

[16]  Sriram Sankar,et al.  Specifying and Testing Software Components using ADL , 1994 .

[17]  Claudio Demartini,et al.  A deadlock detection tool for concurrent Java programs , 1999, Softw. Pract. Exp..

[18]  K. Rustan M. Leino,et al.  Extended static checking , 1998, PROCOMET.

[19]  Sarfraz Khurshid,et al.  Checking Java implementation of a naming architecture using Testera , 2001, Workshop on Software Model Checking @ CAV.

[20]  William Adjie-Winoto,et al.  The design and implementation of an intentional naming system , 2000, OPSR.

[21]  Hans-Martin Hörcher,et al.  Improving Software Tests Using Z Specifications , 1995, ZUM.

[22]  Klaus Havelund,et al.  Model checking programs , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[23]  Kenneth L. McMillan,et al.  Symbolic model checking , 1992 .

[24]  A. Jefferson Offutt,et al.  Generating Tests from UML Specifications , 1999, UML.

[25]  Ivar Jacobson,et al.  The unified modeling language reference manual , 2010 .

[26]  Debra J. Richardson,et al.  Structural specification-based testing: automated support and experimental evaluation , 1999, ESEC/FSE-7.

[27]  J. Michael Spivey,et al.  The Z notation - a reference manual , 1992, Prentice Hall International Series in Computer Science.

[28]  Gary T. Leavens,et al.  A Simple and Practical Approach to Unit Testing: The JML and JUnit Way , 2002, ECOOP.

[29]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[30]  Elliot Schwartz,et al.  Design and implementation of intentional names , 1999 .

[31]  Daniel Jackson,et al.  Finding bugs with a constraint solver , 2000, ISSTA '00.