RepOK‐based reduction of bounded exhaustive testing

While the effectiveness of bounded exhaustive test suites increases as one increases the scope for the bounded exhaustive generation, both the time for test generation and the time for test execution grow exponentially with respect to the scope. In this article, a set of techniques for reducing the time for bounded exhaustive testing, by either reducing the generation time or reducing the obtained bounded exhaustive suites, is proposed. The representation invariant of the software under test's input, implemented as a repOK routine, is exploited for these reductions in two ways: (i) to factor out separate representation invariants for disjoint structures of the inputs; and (ii) to partition valid inputs into equivalence classes, according to how these exercise the repOK code. The first is used in order to split the test input generation process, as disjoint substructures can be independently generated. The second is used in order to reduce the size of a bounded exhaustive test suite, by removing from the suite those tests that are equivalent to some tests already present in the suite. Copyright © 2014 John Wiley & Sons, Ltd.

[1]  Gary T. Leavens,et al.  Beyond Assertions: Advanced Specification and Verification with JML and ESC/Java2 , 2005, FMCO.

[2]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[3]  Sarfraz Khurshid,et al.  Software assurance by bounded exhaustive testing , 2004, IEEE Transactions on Software Engineering.

[4]  Mary Jean Harrold,et al.  An empirical study of the effects of test-suite reduction on fault localization , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[5]  Margus Veanes,et al.  Model-Based Testing with AsmL .NET , 2003 .

[6]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[7]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[8]  Pankaj Jalote,et al.  An Integrated Approach to Software Engineering , 1991, Springer Compass International.

[9]  Sarfraz Khurshid,et al.  Korat: A Tool for Generating Structurally Complex Test Inputs , 2007, 29th International Conference on Software Engineering (ICSE'07).

[10]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[11]  Sarfraz Khurshid,et al.  Parallel test generation and execution with Korat , 2007, ESEC-FSE '07.

[12]  Nazareno Aguirre,et al.  Incorporating Coverage Criteria in Bounded Exhaustive Black Box Test Generation of Structural Inputs , 2011, TAP@TOOLS.

[13]  Glenford J. Myers,et al.  The art of software testing (2. ed.) , 2004 .

[14]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[15]  Daniel Jackson,et al.  Software Abstractions - Logic, Language, and Analysis , 2006 .

[16]  Sarfraz Khurshid,et al.  An Empirical Study of Structural Constraint Solving Techniques , 2009, ICFEM.

[17]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[18]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[19]  Nazareno Aguirre,et al.  Improving Test Generation under Rich Contracts by Tight Bounds and Incremental SAT Solving , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[20]  Sarfraz Khurshid,et al.  TestEra: Specification-Based Testing of Java Programs Using SAT , 2004, Automated Software Engineering.

[21]  Barbara Liskov,et al.  Program Development in Java - Abstraction, Specification, and Object-Oriented Design , 1986 .

[22]  Darko Marinov,et al.  Using Coverage Criteria on RepOK to Reduce Bounded-Exhaustive Test Suites , 2012, TAP@TOOLS.

[23]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[24]  Sarfraz Khurshid,et al.  Test generation through programming in UDITA , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[25]  Gregg Rothermel,et al.  The impact of test suite granularity on the cost-effectiveness of regression testing , 2002, ICSE '02.

[26]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[27]  Darko Marinov,et al.  Reducing the Costs of Bounded-Exhaustive Testing , 2009, FASE.