Do automated program repair techniques repair hard and important bugs?

Automated program repair techniques use a buggy program and a partial specification (typically a test suite) to produce a program variant that satisfies the specification. While prior work has studied patch quality [10, 11] and maintainability [2], it has not examined whether automated repair techniques are capable of repairing defects that developers consider important or that are hard for developers to repair manually. This paper tackles those questions.

[1]  Fan Long,et al.  Automatic patch generation by learning correct code , 2016, POPL.

[2]  Mark Harman,et al.  Genetic Improvement of Software: A Comprehensive Survey , 2018, IEEE Transactions on Evolutionary Computation.

[3]  Irwin Brown,et al.  Profiling grounded theory approaches in information systems research , 2013, Eur. J. Inf. Syst..

[4]  Sarfraz Khurshid,et al.  Specification-Based Program Repair Using SAT , 2011, TACAS.

[5]  Eric Lahtinen,et al.  Automatic error elimination by horizontal code transfer across multiple applications , 2015, PLDI.

[6]  Yuriy Brun,et al.  Data debugging with continuous testing , 2013, ESEC/FSE 2013.

[7]  Abhik Roychoudhury,et al.  Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[8]  Stephen McCamant,et al.  Inference and enforcement of data structure consistency specifications , 2006, ISSTA '06.

[9]  Claire Le Goues,et al.  GenProg: A Generic Method for Automatic Software Repair , 2012, IEEE Transactions on Software Engineering.

[10]  Fan Long,et al.  An Analysis of the Search Spaces for Generate and Validate Patch Generation Systems , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[11]  Yuriy Brun,et al.  Fault and adversary tolerance as an emergent property of distributed systems' software architectures , 2007, EFTS '07.

[12]  Yuriy Brun,et al.  The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs , 2015, IEEE Transactions on Software Engineering.

[13]  Abhik Roychoudhury,et al.  relifix: Automated Repair of Software Regressions , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[14]  Zuohua Ding,et al.  A metamorphic testing approach for supporting program repair without the need for a test oracle , 2017, J. Syst. Softw..

[15]  Fan Long,et al.  An analysis of patch plausibility and correctness for generate-and-validate patch generation systems , 2015, ISSTA.

[16]  Matias Martinez,et al.  Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset , 2016, Empirical Software Engineering.

[17]  Dawei Qi,et al.  SemFix: Program repair via semantic analysis , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[18]  Sarfraz Khurshid,et al.  Specification-Based Test Repair Using a Lightweight Formal Method , 2012, FM.

[19]  Claire Le Goues,et al.  Automatically finding patches using genetic programming , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[20]  Andreas Zeller,et al.  Automated Fixing of Programs with Contracts , 2014 .

[21]  Martin Monperrus,et al.  A critical review of "automatic patch generation learned from human-written patches": essay on the problem statement and the evaluation of automatic software repair , 2014, ICSE.

[22]  W. Eric Wong,et al.  Using Mutation to Automatically Suggest Fixes for Faulty Programs , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[23]  Abhik Roychoudhury,et al.  DirectFix: Looking for Simple Program Repairs , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[24]  K. Charmaz,et al.  The sage handbook of grounded theory , 2007 .

[25]  David Lo,et al.  A Deeper Look into Bug Fixes: Patterns, Replacements, Deletions, and Additions , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[26]  Yuriy Brun,et al.  Do automated program repair techniques repair hard and important bugs? , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[27]  Yuhua Qi,et al.  Efficient Automated Program Repair through Fault-Recorded Testing Prioritization , 2013, 2013 IEEE International Conference on Software Maintenance.

[28]  Rajiv Gupta,et al.  BugFix: A learning-based tool to assist developers in fixing bugs , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[29]  Claire Le Goues,et al.  A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[30]  Tevfik Bultan,et al.  Semantic differential repair for input validation and sanitization , 2014, ISSTA 2014.

[31]  Yuriy Brun,et al.  Is the cure worse than the disease? overfitting in automated program repair , 2015, ESEC/SIGSOFT FSE.

[32]  Shan Lu,et al.  Automated atomicity-violation fixing , 2011, PLDI '11.

[33]  Alexandra Meliou,et al.  Data X-Ray: A Diagnostic Tool for Data Errors , 2015, SIGMOD Conference.

[34]  Yuriy Brun,et al.  Preventing data errors with continuous testing , 2015, ISSTA.

[35]  Yuriy Brun,et al.  Repairing Programs with Semantic Code Search (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[36]  Martin C. Rinard,et al.  Detecting and Escaping Infinite Loops with Jolt , 2011, ECOOP.

[37]  Martin Monperrus,et al.  Automatic repair of buggy if conditions and missing preconditions with SMT , 2014, CSTVA 2014.

[38]  Alessandra Gorla,et al.  Automatic workarounds for web applications , 2010, FSE '10.

[39]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[40]  Yuriy Brun,et al.  Self-Adapting Reliability in Distributed Software Systems , 2015, IEEE Transactions on Software Engineering.

[41]  Matias Martinez,et al.  Automatic Repair of Real Bugs: An Experience Report on the Defects4J Dataset , 2015, ArXiv.

[42]  Bixin Li,et al.  Experience report: How do techniques, programs, and tests impact automated program repair? , 2015, 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE).

[43]  Angelos D. Keromytis,et al.  Countering network worms through automatic patch generation , 2005, IEEE Security & Privacy Magazine.

[44]  Michael D. Ernst,et al.  Defects4J: a database of existing faults to enable controlled testing studies for Java programs , 2014, ISSTA 2014.

[45]  Fan Long,et al.  Staged program repair with condition synthesis , 2015, ESEC/SIGSOFT FSE.

[46]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[47]  David Lo,et al.  S3: syntax- and semantic-guided repair synthesis via programming by examples , 2017, ESEC/SIGSOFT FSE.

[48]  Premkumar T. Devanbu,et al.  Evolution vs. Intelligent Design in Program Patching , 2013 .

[49]  Westley Weimer,et al.  Leveraging program equivalence for adaptive program repair: Models and first results , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[50]  Yuriy Brun,et al.  Fairness testing: testing software for discrimination , 2017, ESEC/SIGSOFT FSE.

[51]  Xin Yao,et al.  A novel co-evolutionary approach to automatic software bug fixing , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[52]  Sarfraz Khurshid,et al.  Juzi: a tool for repairing complex data structures , 2008, ICSE.

[53]  Le Roy,et al.  Research methods in political science : an introduction using MicroCase , 2009 .

[54]  Mark Harman,et al.  API-Constrained Genetic Improvement , 2016, SSBSE.

[55]  Yuriy Brun,et al.  An Architectural Style for Solving Computationally Intensive Problems on Large Networks , 2007, International Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '07).

[56]  Charles Zhang,et al.  Axis: Automatically fixing atomicity violations through solving control constraints , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[57]  Andreas Zeller,et al.  Generating Fixes from Object Behavior Anomalies , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[58]  Charles Zhang,et al.  Grail: context-aware fixing of concurrency bugs , 2014, SIGSOFT FSE.

[59]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[60]  Roger Newson,et al.  Parameters behind “Nonparametric” Statistics: Kendall's tau, Somers’ D and Median Differences , 2002 .

[61]  Jaechang Nam,et al.  Automatic patch generation learned from human-written patches , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[62]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[63]  Alessandra Gorla,et al.  Automatic recovery from runtime failures , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[64]  Yuriy Brun,et al.  Tortoise: Interactive system configuration repair , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[65]  Daniel R. Tauritz,et al.  Multi-objective coevolutionary automated software correction , 2012, GECCO '12.

[66]  Westley Weimer,et al.  A human study of patch maintainability , 2012, ISSTA 2012.

[67]  Mark Harman,et al.  The Current State and Future of Search Based Software Engineering , 2007, Future of Software Engineering (FOSE '07).

[68]  C. Ferguson.  An effect size primer: A guide for clinicians and researchers. , 2009 .

[69]  Zack Coker,et al.  Program transformations to fix C integers , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[70]  Moshe Sipper,et al.  Flight of the FINCH Through the Java Wilderness , 2011, IEEE Transactions on Evolutionary Computation.

[71]  Michael D. Ernst,et al.  Automatically patching errors in deployed software , 2009, SOSP '09.

[72]  K. Charmaz,et al.  Constructing Grounded Theory: A practical guide through qualitative analysis Kathy Charmaz Constructing Grounded Theory: A practical guide through qualitative analysis Sage 224 £19.99 0761973532 0761973532 [Formula: see text]. , 2006, Nurse researcher.

[73]  Westley Weimer,et al.  Post-compiler software optimization for reducing energy , 2014, ASPLOS.