Improving automation in developer testing: State of the practice

Developer testing, a common step in software development, involves generating desirable test inputs and checking the behavior of the program unit under test during the execution of the test inputs. Existing industrial developer testing tools include various techniques to address challenges of generating desirable test inputs and checking the behavior of the program unit under test. This paper presents an overview of techniques implemented in industrial developer testing tools to address challenges in improving automation in developer testing. These techniques are summarized from two main aspects: test efficiency (e.g., with a focus on cost) and test effectiveness (e.g., with a focus on benefit).

[1]  Michael C. Feathers Working Effectively with Legacy Code , 2004, XP/Agile Universe.

[2]  David Notkin,et al.  Rostra: a framework for detecting redundant object-oriented unit tests , 2004 .

[3]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[4]  簡聰富,et al.  物件導向軟體之架構(Object-Oriented Software Construction)探討 , 1989 .

[5]  David Notkin,et al.  Checking inside the black box: regression testing by comparing value spectra , 2005, IEEE Transactions on Software Engineering.

[6]  Matthew B. Dwyer,et al.  Carving differential unit test cases from system test cases , 2006, SIGSOFT '06/FSE-14.

[7]  Michael D. Ernst,et al.  Automatic test factoring for java , 2005, ASE '05.

[8]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[9]  Nikolai Tillmann,et al.  Mock-object generation with behavior , 2006, 21st IEEE/ACM International Conference on Automated Software Engineering (ASE'06).

[10]  Tao Xie,et al.  Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[11]  Nikolai Tillmann,et al.  Unit tests reloaded: parameterized unit testing with symbolic execution , 2006, IEEE Software.

[12]  Marat Boshernitsan,et al.  From daikon to agitator: lessons and challenges in building a commercial tool for developer testing , 2006, ISSTA '06.

[13]  David Notkin,et al.  Tool-assisted unit-test generation and selection based on operational abstractions , 2006, Automated Software Engineering.

[14]  Alessandro Orso,et al.  Selective capture and replay of program executions , 2005, WODA '05.

[15]  Tao Xie,et al.  Augmenting Automatically Generated Unit-Test Suites with Regression Oracle Checking , 2006, ECOOP.

[16]  Nikolai Tillmann,et al.  Parameterized unit tests , 2005, ESEC/FSE-13.

[17]  Tao Xie,et al.  UnitPlus: assisting developer testing in Eclipse , 2007, eclipse '07.