Evolutionary repair of faulty software

Abstract: Testing and fault localization are very expensive software engineering tasks that have been tried to be automated. Although many successful techniques have been designed, the actual change of the code for fixing the discovered faults is still a human-only task. Even in the ideal case in which automated tools could tell us exactly where the location of a fault is, it is not always trivial how to fix the code. In this paper we analyse the possibility of automating the complex task of fixing faults. We propose to model this task as a search problem, and hence to use for example evolutionary algorithms to solve it. We then discuss the potential of this approach and how its current limitations can be addressed in the future. This task is extremely challenging and mainly unexplored in the literature. Hence, this paper only covers an initial investigation and gives directions for future work. A research prototype called JAFF and a case study are presented to give first validation of this approach.

[1]  Claire Le Goues,et al.  A genetic programming approach to automated software repair , 2009, GECCO.

[2]  Roderick Bloem,et al.  Finding and fixing faults , 2005, J. Comput. Syst. Sci..

[3]  Mary Jean Harrold,et al.  An empirical study of the effects of test-suite reduction on fault localization , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[4]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[5]  Alessandro Orso,et al.  A technique for dynamic updating of Java software , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[6]  Louise A. Dennis,et al.  Proof-Directed Debugging and Repair , 2006 .

[7]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[8]  Moshe Sipper,et al.  Genetic programming in the wild: evolving unrestricted bytecode , 2009, GECCO.

[9]  Jeffrey S. Foster,et al.  A comparison of bug finding tools for Java , 2004, 15th International Symposium on Software Reliability Engineering.

[10]  Roderick Bloem,et al.  Program Repair as a Game , 2005, CAV.

[11]  Tao Xie,et al.  Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[12]  David Hovemeyer,et al.  Finding bugs is easy , 2004, SIGP.

[13]  Xiangyu Zhang,et al.  A study of effectiveness of dynamic slicing in locating real faults , 2006, Empirical Software Engineering.

[14]  Andrea Arcuri,et al.  On the automation of fixing software bugs , 2008, ICSE Companion '08.

[15]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[16]  Westley Weimer,et al.  Patches as better bug reports , 2006, GPCE '06.

[17]  Yuanyuan Zhou,et al.  CP-Miner: finding copy-paste and related bugs in large-scale software code , 2006, IEEE Transactions on Software Engineering.

[18]  Claire Le Goues,et al.  Using Execution Paths to Evolve Software Patches , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[19]  Jeffrey O. Kephart,et al.  The Vision of Autonomic Computing , 2003, Computer.

[20]  Brian Demsky Data structure repair using goal-directed reasoning , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[21]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[22]  Andrea Arcuri,et al.  It Does Matter How You Normalise the Branch Distance in Search Based Software Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[23]  Mark Harman,et al.  Reformulating software engineering as a search problem , 2003 .

[24]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[25]  Chadd C. Williams,et al.  Automatic mining of source code repositories to improve bug finding techniques , 2005, IEEE Transactions on Software Engineering.

[26]  Markus Stumptner,et al.  Thirteenth International Workshop on Principles of Diagnosis (DX-2002) , 2002 .

[27]  Andreas Zeller Automated Debugging: Are We Close , 2001, Computer.

[28]  Paul W. H. Chung,et al.  Proceedings of the Sixth International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems , 1993 .

[29]  Joseph Robert Horgan,et al.  Dynamic program slicing , 1990, PLDI '90.

[30]  H. Cleve,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[31]  Lei Zhao,et al.  A Crosstab-based Statistical Method for Effective Fault Localization , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[32]  Yue Jia,et al.  MILU: A Customizable, Runtime-Optimized Higher Order Mutation Testing Tool for the Full C Language , 2008, Testing: Academic & Industrial Conference - Practice and Research Techniques (taic part 2008).

[33]  Jan Madey,et al.  The Z notation: A reference manual , 1990 .

[34]  Xin Yao,et al.  Search based software testing of object-oriented containers , 2008, Inf. Sci..

[35]  John A. Clark,et al.  Multi-objective Improvement of Software Using Co-evolution and Smart Seeding , 2008, SEAL.

[36]  J. Michael Spivey,et al.  The Z notation - a reference manual , 1992, Prentice Hall International Series in Computer Science.

[37]  Riccardo Poli,et al.  Foundations of Genetic Programming , 1999, Springer Berlin Heidelberg.

[38]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[39]  Simon M. Lucas,et al.  Evolving Modular Recursive Sorting Algorithms , 2007, EuroGP.

[40]  Markus Stumptner,et al.  Model-Based Program Debugging and Repair , 1996, IEA/AIE.

[41]  Mireille Ducassé,et al.  A Pragmatic Survey of Automated Debugging , 1993, AADEBUG.

[42]  Yan Zhang,et al.  CTL Model Update for System Modifications , 2011, J. Artif. Intell. Res..

[43]  Markus Stumptner,et al.  A Survey of Intelligent Debugging , 1998, AI Commun..

[44]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[45]  Cemal Yilmaz,et al.  An automated model-based debugging approach , 2007, ASE.

[46]  MaYu-Seung,et al.  MuJava: an automated class mutation system , 2005 .

[47]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[48]  Xin Yao,et al.  A novel co-evolutionary approach to automatic software bug fixing , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[49]  Peter Fritzson,et al.  Generalized algorithmic debugging and testing , 1991, LOPL.

[50]  Lionel C. Briand,et al.  Using Machine Learning to Support Debugging with Tarantula , 2007, The 18th IEEE International Symposium on Software Reliability (ISSRE '07).

[51]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[52]  Georg Gottlob,et al.  Enhancing Model Checking in Verification by AI Techniques , 1999, Artif. Intell..

[53]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[54]  Farn Wang,et al.  Program Repair Suggestions from Graphical State-Transition Specifications , 2008, FORTE.

[55]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[56]  Albert L. Baker,et al.  JML: A Notation for Detailed Design , 1999, Behavioral Specifications of Businesses and Systems.

[57]  Roderick Bloem,et al.  Repair of Boolean Programs with an Application to C , 2006, CAV.

[58]  Dewayne E. Perry,et al.  Toward understanding the rhetoric of small source code changes , 2005, IEEE Transactions on Software Engineering.

[59]  Joseph Robert Horgan,et al.  Fault localization using execution slices and dataflow tests , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[60]  José Antonio Lozano,et al.  Scatter Search in software testing, comparison and collaboration with Estimation of Distribution Algorithms , 2006, Eur. J. Oper. Res..

[61]  James Miller,et al.  On the possibilities of (pseudo-) software cloning from external interactions , 2008, Soft Comput..

[62]  Xin Yao,et al.  Coevolving programs and unit tests from their specification , 2007, ASE.

[63]  A. Zeller Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[64]  Mark Weiser,et al.  Programmers use slices when debugging , 1982, CACM.

[65]  Sean Luke,et al.  A Comparison of Bloat Control Methods for Genetic Programming , 2006, Evolutionary Computation.

[66]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[67]  Bernhard Rumpe,et al.  Behavioral Specifications of Businesses and Systems , 1999, Behavioral Specifications of Businesses and Systems.

[68]  James A. Jones Semi-automatic fault localization , 2008 .

[69]  B. Myers Debugging Reinvented: Asking and Answering Why and Why Not Questions about Program Behavior , 2008 .

[70]  Harmen-Hinrich Sthamer,et al.  The automatic generation of software test data using genetic algorithms , 1995 .

[71]  Jyotirmoy V. Deshmukh,et al.  Automatic Generation of Local Repairs for Boolean Programs , 2008, 2008 Formal Methods in Computer-Aided Design.

[72]  John Nicholls,et al.  Z notation , 1994 .

[73]  Claire Le Goues,et al.  Automatically finding patches using genetic programming , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[74]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[75]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[76]  Thomas Zimmermann,et al.  Extraction of bug localization benchmarks from history , 2007, ASE.

[77]  Louise A. Dennis,et al.  Program Slicing and Middle-Out Reasoning for Error Location and Repair , 2006 .

[78]  Sigrid Eldh Software Testing Techniques , 2007 .

[79]  Foreword to Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[80]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[81]  John A. Clark,et al.  Formulating software engineering as a search problem , 2003, IEE Proc. Softw..

[82]  Per Kristian Lehre,et al.  Theoretical Runtime Analyses of Search Algorithms on the Test Data Generation for the Triangle Classification Problem , 2008, 2008 IEEE International Conference on Software Testing Verification and Validation Workshop.

[83]  Stefan Wagner,et al.  An Evaluation of Two Bug Pattern Tools for Java , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[84]  Mark Harman,et al.  Testability transformation , 2004, IEEE Transactions on Software Engineering.

[85]  Neelam Gupta,et al.  Automated Debugging Using Path-Based Weakest Preconditions , 2004, FASE.

[86]  Günther Ruhe,et al.  Search Based Software Engineering , 2013, Lecture Notes in Computer Science.

[87]  Lionel C. Briand,et al.  Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria , 2006, IEEE Transactions on Software Engineering.

[88]  Perdita Stevens,et al.  Modelling Recursive Calls with UML State Diagrams , 2003, FASE.

[89]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[90]  Cyrille Artho,et al.  Iterative delta debugging , 2009, International Journal on Software Tools for Technology Transfer.

[91]  Kerstin Eder,et al.  Improving XCS performance on overlapping binary problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[92]  Markus Stumptner,et al.  A model-based approach to software debugging , 1996 .