eXpress : Guided Path Exploration for Regression Test Generation

Regression test generation aims at generating a test suite that can detect behavioral differences between the original and the new versions of a program. Regression test generation can be automated by using Dynamic Symbolic Execution (DSE), a state-of-the-art test generation technique. DSE explores all feasible paths in the program but exploration of all these paths can often be expensive. However, if our aim is to detect behavioral differences between two versions of a program, we do not need to explore all these paths in the program, since not all these paths are relevant for detecting behavioral differences. In this paper, we propose an approach on guided path exploration that avoids exploring irrelevant paths in terms of detecting behavioral differences. Hence, behavioral differences are more likely to be detected earlier in path exploration. In addition, our approach leverages the existing test suite (if available) for the original version to efficiently execute the changed regions of the program and infect program states. Experimental results on 67 versions (in total) of four programs show that our approach requires about 36% fewer amount of time (on average) to detect behavioral differences than exploration without using our approach. In addition, our approach detects 6 behavioral difference that could not be detected by exploration without using our approach (within a time bound). Furthermore, our approach requires 67% fewer amount of time to find behavioral differences by exploiting an existing test suite than exploration without using the test suite.

[1]  Zhihong Xu,et al.  Directed test suite augmentation , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[2]  Alessandro Orso,et al.  Automated Behavioral Regression Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[3]  Nikolai Tillmann,et al.  Fitness-guided path exploration in dynamic symbolic execution , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[4]  Nikolai Tillmann,et al.  Guided path exploration for regression test generation , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[5]  Matthew B. Dwyer,et al.  Carving and Replaying Differential Unit Test Cases from System Test Cases , 2009, IEEE Transactions on Software Engineering.

[6]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[7]  Matthew B. Dwyer,et al.  Differential symbolic execution , 2008, SIGSOFT '08/FSE-16.

[8]  Alessandro Orso,et al.  Test-Suite Augmentation for Evolving Software , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[9]  Tao Xie,et al.  DiffGen: Automated Regression Unit-Test Generation , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[10]  Koushik Sen,et al.  Heuristics for Scalable Dynamic Test Generation , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[11]  Xiangyu Zhang,et al.  Efficient program execution indexing , 2008, PLDI '08.

[12]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[13]  Nikolai Tillmann,et al.  Demand-Driven Compositional Symbolic Execution , 2008, TACAS.

[14]  Koushik Sen,et al.  Predictive testing: amplifying the effectiveness of software testing , 2007, ESEC-FSE '07.

[15]  R. Evans,et al.  Differential testing: a new approach to change detection , 2007, ESEC-FSE '07.

[16]  Rupak Majumdar,et al.  Hybrid Concolic Testing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[17]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[18]  Alessandro Orso,et al.  MATRIX: Maintenance-Oriented Testing Requirements Identifier and Examiner , 2006, Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART'06).

[19]  Ralph E. Johnson,et al.  Automated Detection of Refactorings in Evolving Components , 2006, ECOOP.

[20]  J. Jenny Li,et al.  Prioritize code for testing to improve code coverage of complex software , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[21]  Michael D. Ernst,et al.  Automatic test factoring for java , 2005, ASE '05.

[22]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[23]  David Notkin,et al.  Checking inside the black box: regression testing by comparing value spectra , 2005, IEEE Transactions on Software Engineering.

[24]  Nikolai Tillmann,et al.  Parameterized unit tests , 2005, ESEC/FSE-13.

[25]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[26]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[27]  Alessandro Orso,et al.  Selective capture and replay of program executions , 2005, WODA '05.

[28]  Xiaoxia Ren,et al.  Chianti: a change impact analysis tool for Java programs , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[29]  D. Notkin,et al.  Rostra: a framework for detecting redundant object-oriented unit tests , 2004, Proceedings. 19th International Conference on Automated Software Engineering, 2004..

[30]  Gregg Rothermel,et al.  Whole program path-based dynamic impact analysis , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[31]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[32]  Susan Horwitz Tool Support for Improving Test Coverage , 2002, ESOP.

[33]  Gregg Rothermel,et al.  A safe, efficient regression test selection technique , 1997, TSEM.

[34]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..

[35]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[36]  Lori A. Clarke,et al.  A System to Generate Test Data and Symbolically Execute Programs , 1976, IEEE Transactions on Software Engineering.