Live feedback on behavioral changes

The costs to find and fix bugs grows over time, to the point where fixing a bug after release may cost as much as 100 times more than before release. To help programmers find bugs as soon as they are introduced, we sketch a plugin for an integrated development environment that provides live feedback about behavioral changes to Java programs by continuously generating tests, running the tests on the current and previous versions of the program, and comparing the results. Such a tool would allow programmers to better understand how their changes affect the behavior of their programs. As a proof of concept, we developed a prototype that found a bug that remained undetected by pair programmers working on JHotDraw in a previous study. Had the programmers performed this change with our plugin, they would have been notified about the bug as soon as they introduced it.

[1]  Rohit Gheyi,et al.  Making Program Refactoring Safer , 2010, IEEE Software.

[2]  Zhendong Su,et al.  Synthesizing method sequences for high-coverage testing , 2011, OOPSLA '11.

[3]  Gustavo Soares Soares.  Automated behavioral testing of refactoring engines , 2012, SPLASH '12.

[4]  Barry W. Boehm,et al.  Software Defect Reduction Top 10 List , 2001, Computer.

[5]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[6]  Shuvendu K. Lahiri,et al.  SYMDIFF: A Language-Agnostic Semantic Diff Tool for Imperative Programs , 2012, CAV.

[7]  Emerson R. Murphy-Hill,et al.  Reconciling manual and automatic refactoring , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[8]  Miryung Kim,et al.  A field study of refactoring challenges and benefits , 2012, SIGSOFT FSE.

[9]  Barbara G. Ryder,et al.  Tool Support for Change-Centric Test Development , 2010, IEEE Software.

[10]  Alessandro Orso,et al.  Automated Behavioral Regression Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[11]  Michael D. Ernst,et al.  An experimental evaluation of continuous testing during development , 2004, ISSTA '04.

[12]  Miryung Kim,et al.  An empirical investigation into the impact of refactoring on regression testing , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).