Check 'n' crash: combining static checking and testing

We present an automatic error-detection approach that combines static checking and concrete test-case generation. Our approach consists of taking the abstract error conditions inferred using theorem proving techniques by a static checker (ESC/Java), deriving specific error conditions using a constraint solver, and producing concrete test cases (with the JCrasher tool) that are executed to determine whether an error truly exists. The combined technique has advantages over both static checking and automatic testing individually. Compared to ESC/Java, we eliminate spurious warnings and improve the ease-of-comprehension of error reports through the production of Java counterexamples. Compared to JCrasher, we eliminate the blind search of the input space, thus reducing the testing time and increasing the test quality.

[1]  Albert L. Baker,et al.  Preliminary design of JML: a behavioral interface specification language for java , 2006, SOEN.

[2]  David Detlefs,et al.  Simplify: a theorem prover for program checking , 2005, JACM.

[3]  K. Rustan M. Leino,et al.  Efficient weakest preconditions , 2005, Inf. Process. Lett..

[4]  David R. Cok,et al.  ESC/Java2: Uniting ESC/Java and JML Progress and Issues in Building and Using ESC/Java2, Including a Case Study Involving the Use of the Tool to Verify Portions of an Internet Voting Tally System , 2005 .

[5]  W. Pugh,et al.  Finding bugs is easy , 2004, SIGP.

[6]  Alex Groce,et al.  Counterexample Guided Abstraction Refinement Via Program Execution , 2004, ICFEM.

[7]  Jeffrey S. Foster,et al.  A comparison of bug finding tools for Java , 2004, 15th International Symposium on Software Reliability Engineering.

[8]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[9]  Thomas A. Henzinger,et al.  Generating tests from counterexamples , 2004, Proceedings. 26th International Conference on Software Engineering.

[10]  David Notkin,et al.  Tool-assisted unit test selection based on operational violations , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[11]  Dawson R. Engler,et al.  Some Lessons from Using Static Analysis and Software Model Checking for Bug Finding , 2003, SoftMC@CAV.

[12]  Dawson R. Engler,et al.  Z-Ranking: Using Statistical Analysis to Counter the Impact of Static Analysis Approximations , 2003, SAS.

[13]  Daniel Jackson,et al.  Checking Properties of Heap-Manipulating Procedures with a Constraint Solver , 2003, TACAS.

[14]  D. Engler,et al.  Using redundancies to find errors , 2003, SOEN.

[15]  P. David Stotts,et al.  An Informal Formal Method for Systematic JUnit Test Case Generation , 2002, XP/Agile Universe.

[16]  Steve Loughran,et al.  Java Development with Ant , 2002 .

[17]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[18]  Georg Ringwelski,et al.  POOC: A Platform for Object-Oriented Constraint Programming , 2002, International Workshop on Constraint Solving and Constraint Logic Programming.

[19]  Greg Nelson,et al.  Extended static checking for Java , 2002, PLDI '02.

[20]  Dawson R. Engler,et al.  A system and language for building system-specific, static analyses , 2002, PLDI '02.

[21]  K. Rustan M. Leino,et al.  Houdini, an Annotation Assistant for ESC/Java , 2001, FME.

[22]  Daniel Jackson,et al.  Finding bugs with a constraint solver , 2000, ISSTA '00.

[23]  Kent Beck,et al.  Test-infected: programmers love writing tests , 2000 .

[24]  K. Rustan M. Leino,et al.  ESC/Java User's Manual , 2000 .

[25]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[26]  Bjarne Steensgaard,et al.  Points-to analysis in almost linear time , 1996, POPL '96.

[27]  D. B. Davis,et al.  Sun Microsystems Inc. , 1993 .