A Survey on Automated Program Repair Techniques

With the rapid development and large-scale popularity of program software, modern society increasingly relies on software systems. However, the problems exposed by software have also come to the fore. Software defect has become an important factor troubling developers. In this context, Automated Program Repair (APR) techniques have emerged, aiming to automatically fix software defect problems and reduce manual debugging work. In particular, benefiting from the advances in deep learning, numerous learning-based APR techniques have emerged in recent years, which also bring new opportunities for APR research. To give researchers a quick overview of APR techniques' complete development and future opportunities, we revisit the evolution of APR techniques and discuss in depth the latest advances in APR research. In this paper, the development of APR techniques is introduced in terms of four different patch generation schemes: search-based, constraint-based, template-based, and learning-based. Moreover, we propose a uniform set of criteria to review and compare each APR tool, summarize the advantages and disadvantages of APR techniques, and discuss the current state of APR development. Furthermore, we introduce the research on the related technical areas of APR that have also provided a strong motivation to advance APR development. Finally, we analyze current challenges and future directions, especially highlighting the critical opportunities that large language models bring to APR research.

[1]  Hiroaki Hayashi,et al.  Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing , 2021, ACM Comput. Surv..

[2]  Martin Monperrus,et al.  Neural Transfer Learning for Repairing Security Vulnerabilities in C Code , 2021, IEEE Transactions on Software Engineering.

[3]  Ting Liu,et al.  SeqTrans: Automatic Vulnerability Fix Via Sequence to Sequence Learning , 2020, IEEE Transactions on Software Engineering.

[4]  X. Franch,et al.  Energy Consumption of Automated Program Repair , 2022, ArXiv.

[5]  Dinh Q. Phung,et al.  VulRepair: a T5-based automated software vulnerability repair , 2022, ESEC/SIGSOFT FSE.

[6]  Lingming Zhang,et al.  Practical Program Repair in the Era of Large Pre-trained Language Models , 2022, ArXiv.

[7]  Xiaofei Xie,et al.  TransRepair: Context-aware Program Repair for Compilation Errors , 2022, ASE.

[8]  Sumit Gulwani,et al.  Repairing Bugs in Python Assignments Using Large Language Models , 2022, ArXiv.

[9]  Sumit Gulwani,et al.  Repair Is Nearly Generation: Multilingual Program Repair with LLMs , 2022, AAAI.

[10]  T. Bryksin,et al.  Out of the BLEU: how should we assess quality of the Code Generation models? , 2022, J. Syst. Softw..

[11]  Lingming Zhang,et al.  An extensive study on pre-trained models for program understanding and generation , 2022, ISSTA.

[12]  Lingming Zhang,et al.  Less training, more repairing please: revisiting automated program repair via zero-shot learning , 2022, ESEC/SIGSOFT FSE.

[13]  Nguyen Quoc Viet Hung,et al.  CIRCLE: continual repair across programming languages , 2022, ISSTA.

[14]  Shin Hwei Tan,et al.  Automated Repair of Programs from Large Language Models , 2022, 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE).

[15]  Xudong Liu,et al.  Improving Fault Localization and Program Repair with Deep Semantic Features and Transferred Knowledge , 2022, 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE).

[16]  T. Nguyen,et al.  DEAR: A Novel Deep Learning-based Approach for Automated Program Repair , 2022, 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE).

[17]  Frank F. Xu,et al.  A systematic evaluation of large language models of code , 2022, MAPS@PLDI.

[18]  Monperrus Martin,et al.  Neural Program Repair with Execution-based Backpropagation , 2021, 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE).

[19]  Anindya Iqbal,et al.  Review4Repair: Code Review Aided Automatic Program Repairing , 2020, Inf. Softw. Technol..

[20]  Timothy M. Hospedales,et al.  Meta-Learning in Neural Networks: A Survey , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ahmed E. Hassan,et al.  Just-In-Time Defect Identification and Localization: A Two-Phase Framework , 2020, IEEE Transactions on Software Engineering.

[22]  Tian Zhang,et al.  Restore: Retrospective Fault Localization Enhancing Automated Program Repair , 2019, IEEE Transactions on Software Engineering.

[23]  Brendan Dolan-Gavitt,et al.  Examining Zero-Shot Vulnerability Repair with Large Language Models , 2021, 2023 IEEE Symposium on Security and Privacy (SP).

[24]  Yuriy Brun,et al.  SOSRepair: Expressive Semantic Search for Real-World Program Repair , 2021, IEEE Transactions on Software Engineering.

[25]  Yue Wang,et al.  CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation , 2021, EMNLP.

[26]  Claire Le Goues,et al.  VarFix: balancing edit expressiveness and search effectiveness in automated program repair , 2021, ESEC/SIGSOFT FSE.

[27]  Lu Zhang,et al.  A syntax-guided edit decoder for neural program repair , 2021, ESEC/SIGSOFT FSE.

[28]  Percy Liang,et al.  Break-It-Fix-It: Unsupervised Learning for Program Repair , 2021, ICML.

[29]  Justyna Petke,et al.  Refining Fitness Functions for Search-Based Program Repair , 2021, 2021 IEEE/ACM International Workshop on Automated Program Repair (APR).

[30]  Ahmad Baraani-Dastjerdi,et al.  CGenProg: Adaptation of cartesian genetic programming with migration and opposite guesses for automatic repair of software regression faults , 2021, Expert Syst. Appl..

[31]  Manish Motwani,et al.  High-Quality Automated Program Repair , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion).

[32]  Xia Li,et al.  Self-Boosted Automated Program Repair , 2021, ArXiv.

[33]  André Restivo,et al.  Automatic Program Repair as Semantic Suggestions: An Empirical Study , 2021, 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST).

[34]  Gary T. Leavens,et al.  Exploring True Test Overfitting in Dynamic Automated Program Repair using Formal Methods , 2021, 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST).

[35]  Eunseok Lee,et al.  Are datasets for information retrieval-based bug localization techniques trustworthy? , 2021, Empirical Software Engineering.

[36]  David Lo,et al.  Is the Ground Truth Really Accurate? Dataset Purification for Automated Program Repair , 2021, 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER).

[37]  Xiaoguang Mao,et al.  On the Impact of Flaky Tests in Automated Program Repair , 2021, 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER).

[38]  Thibaud Lutellier,et al.  CURE: Code-Aware Neural Machine Translation for Automatic Program Repair , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).

[39]  Jacques Klein,et al.  A critical review on the evaluation of automated program repair systems , 2021, J. Syst. Softw..

[40]  Shin Hwei Tan,et al.  Automated Patch Transplantation , 2020, ACM Trans. Softw. Eng. Methodol..

[41]  Monperrus Martin,et al.  A Software-Repair Robot Based on Continual Learning , 2020, IEEE Software.

[42]  Ming Zhou,et al.  GraphCodeBERT: Pre-training Code Representations with Data Flow , 2020, ICLR.

[43]  Lingming Zhang,et al.  Fast and Precise On-the-Fly Patch Validation for All , 2020, 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).

[44]  Matias Martinez,et al.  E-APR: Mapping the effectiveness of automated program repair techniques , 2020, Empirical Software Engineering.

[45]  Yu Pei,et al.  Contract-Based Program Repair Without The Contracts: An Extended Study , 2020, IEEE Transactions on Software Engineering.

[46]  Matias Martinez,et al.  Automated patch assessment for program repair at scale , 2019, Empirical Software Engineering.

[47]  Mario Fritz,et al.  SampleFix: Learning to Generate Functionally Diverse Fixes , 2019, PKDD/ECML Workshops.

[48]  Denys Poshyvanyk,et al.  SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair , 2018, IEEE Transactions on Software Engineering.

[49]  Shin Yoo,et al.  Mining Fix Patterns for FindBugs Violations , 2017, IEEE Transactions on Software Engineering.

[50]  Martin Vechev,et al.  TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer , 2021, ICML.

[51]  Ramesh Karri,et al.  Can OpenAI Codex and Other Large Language Models Help Us Fix Security Bugs? , 2021, ArXiv.

[52]  Justyna Petke,et al.  Refining Fitness Functions for Search-Based Automated Program Repair - A Case Study with ARJA and ARJA-e , 2021, SSBSE.

[53]  Furu Wei,et al.  Grammar-Based Patches Generation for Automated Program Repair , 2021, FINDINGS.

[54]  Alpana Dubey,et al.  RepairNet: Contextual Sequence-to-Sequence Network for Automated Program Repair , 2021, AIED.

[55]  Xiaoyuan Xie,et al.  Essential Spectrum-based Fault Localization , 2021 .

[56]  Jacques Klein,et al.  Evaluating Representation Learning of Code Changes for Predicting Patch Correctness in Program Repair , 2020, 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[57]  Rui Abreu,et al.  Multiple fault localization of software programs: A systematic literature review , 2020, Inf. Softw. Technol..

[58]  Yitong Li,et al.  CoCoNuT: combining context-aware neural translation models using ensemble for program repair , 2020, ISSTA.

[59]  Xia Li,et al.  Can automated program repair refine fault localization? a unified debugging approach , 2020, ISSTA.

[60]  Shaohua Wang,et al.  DLFix: Context-based Code Transformation Learning for Automated Program Repair , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE).

[61]  Jacques Klein,et al.  On the Efficiency of Test Suite based Program Repair A Systematic Assessment of 16 Automated Repair Systems for Java Programs , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE).

[62]  Ming Wen,et al.  Boosting Automated Program Repair with Bug-Inducing Commits , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER).

[63]  Lihua Liu,et al.  Precfix: Large-Scale Patch Recommendation by Mining Defect-Patch Pairs , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).

[64]  Percy Liang,et al.  Graph-based, Self-Supervised Program Repair from Diagnostic Feedback , 2020, ICML.

[65]  Bo Lin,et al.  Automated Patch Correctness Assessment: How Far are We? , 2020, 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[66]  Ting Liu,et al.  CodeBERT: A Pre-Trained Model for Programming and Natural Languages , 2020, FINDINGS.

[67]  Gabriele Bavota,et al.  On Learning Meaningful Assert Statements for Unit Test Cases , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE).

[68]  Yusuke Nemoto,et al.  Experience Report: How Effective is Automated Program Repair for Industrial Software? , 2020, 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[69]  Marcel Böhme,et al.  Human-In-The-Loop Automatic Program Repair , 2019, 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST).

[70]  Jacques Klein,et al.  FixMiner: Mining relevant fix patterns for automated program repair , 2018, Empirical Software Engineering.

[71]  Wolfgang Banzhaf,et al.  ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programming , 2017, IEEE Transactions on Software Engineering.

[72]  Monperrus Martin Automatic Software Repair: a Bibliography , 2020 .

[73]  Xiang Gao Beyond Tests: Program Vulnerability Repair via Crash Constraint Extraction , 2020 .

[74]  Claire Le Goues,et al.  Automated program repair , 2019, Commun. ACM.

[75]  Rahul Gupta,et al.  Deep Reinforcement Learning for Syntactic Error Repair in Student Programs , 2019, AAAI.

[76]  Jacques Klein,et al.  iFixR: bug report driven program repair , 2019, ESEC/SIGSOFT FSE.

[77]  Rui Abreu,et al.  Empirical review of Java program repair tools: a large-scale experiment on 2,141 bugs and 23,551 repair attempts , 2019, ESEC/SIGSOFT FSE.

[78]  Qi Xin,et al.  Better Code Search and Reuse for Better Program Repair , 2019, 2019 IEEE/ACM International Workshop on Genetic Improvement (GI).

[79]  Seemanta Saha,et al.  Harnessing Evolution for Multi-Hunk Program Repair , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[80]  Mark Harman,et al.  SapFix: Automated End-to-End Repair at Scale , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).

[81]  Aditya Kanade,et al.  Neural Program Repair by Jointly Learning to Localize and Repair , 2019, ICLR.

[82]  Tegawendé F. Bissyandé,et al.  TBar: revisiting template-based automated program repair , 2019, ISSTA.

[83]  Johannes Bader,et al.  Getafix: learning to fix bugs automatically , 2019, Proc. ACM Program. Lang..

[84]  Peter W. O'Hearn,et al.  Separation logic , 2019, Commun. ACM.

[85]  Wei-Ngan Chin,et al.  Automatic Program Repair Using Formal Verification and Expression Templates , 2019, VMCAI.

[86]  Gabriele Bavota,et al.  An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation , 2018, ACM Trans. Softw. Eng. Methodol..

[87]  Tegawendé F. Bissyandé,et al.  AVATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations , 2018, 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[88]  Jacques Klein,et al.  You Cannot Fix What You Cannot Find! An Investigation of Fault Localization Bias in Benchmarking Automated Program Repair Systems , 2018, 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST).

[89]  Lingming Zhang,et al.  Practical program repair via bytecode mutation , 2018, ISSTA.

[90]  David Lo,et al.  On Reliability of Patch Correctness Assessment , 2018, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[91]  Martin White,et al.  Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities , 2017, 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[92]  Hongyu Zhang,et al.  Shaping program repair space with existing patches and similar code , 2018, ISSTA.

[93]  Daniela Micucci,et al.  Automatic Software Repair: A Survey , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[94]  Claire Le Goues,et al.  Static Automated Program Repair for Heap Properties , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[95]  Sarfraz Khurshid,et al.  Towards Practical Program Repair with On-demand Candidate Generation , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[96]  Pushmeet Kohli,et al.  Neuro-Symbolic Program Corrector for Introductory Programming Assignments , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[97]  Ming Wen,et al.  Context-Aware Patch Generation for Better Automated Program Repair , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[98]  Sang Peter Chin,et al.  Learning to Repair Software Vulnerabilities with Generative Adversarial Networks , 2018, NeurIPS.

[99]  Hao Zhong,et al.  Mining stackoverflow for program repair , 2018, 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[100]  Gang Huang,et al.  Identifying Patch Correctness in Test-Based Program Repair , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[101]  Martin Monperrus,et al.  Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs , 2018, IEEE Transactions on Software Engineering.

[102]  David Lo,et al.  Overfitting in semantics-based automated program repair , 2018, Empirical Software Engineering.

[103]  Qi Xin,et al.  Leveraging syntax-related code for automated program repair , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[104]  Carlo A. Furia,et al.  Contract-based program repair without the contracts , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[105]  Hiroaki Yoshida,et al.  Elixir: Effective object-oriented program repair , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[106]  Vern Paxson,et al.  A Large-Scale Empirical Study of Security Patches , 2017, CCS.

[107]  Beijun Shen,et al.  CRSearcher: Searching Code Database for Repairing Bugs , 2017, Internetware.

[108]  David Lo,et al.  S3: syntax- and semantic-guided repair synthesis via programming by examples , 2017, ESEC/SIGSOFT FSE.

[109]  Alexey Zhikhartsev,et al.  Better test cases for better automated program repair , 2017, ESEC/SIGSOFT FSE.

[110]  Fan Long,et al.  Automatic inference of code transforms for patch generation , 2017, ESEC/SIGSOFT FSE.

[111]  Rahul Gupta,et al.  DeepFix: Fixing Common C Language Errors by Deep Learning , 2017, AAAI.

[112]  Orna Grumberg,et al.  Sound and Complete Mutation-Based Program Repair , 2016, FM.

[113]  David Lo,et al.  Enhancing Automated Program Repair with Deductive Verification , 2016, 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[114]  Rui Abreu,et al.  A Survey on Software Fault Localization , 2016, IEEE Transactions on Software Engineering.

[115]  Armando Solar-Lezama,et al.  sk_p: a neural program corrector for MOOCs , 2016, SPLASH.

[116]  Xiaoguang Mao,et al.  Automated Program Repair by Using Similar Code Containing Fix Ingredients , 2016, 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC).

[117]  Abhik Roychoudhury,et al.  Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[118]  Rishabh Singh,et al.  Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks , 2016, ArXiv.

[119]  David Lo,et al.  History Driven Program Repair , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[120]  Fan Long,et al.  Automatic patch generation by learning correct code , 2016, POPL.

[121]  Matias Martinez ASTOR: A Program Repair Library for Java , 2016 .

[122]  Yuriy Brun,et al.  Repairing Programs with Semantic Code Search (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[123]  Yuriy Brun,et al.  Is the cure worse than the disease? overfitting in automated program repair , 2015, ESEC/SIGSOFT FSE.

[124]  Fan Long,et al.  Staged program repair with condition synthesis , 2015, ESEC/SIGSOFT FSE.

[125]  David Lo,et al.  Information retrieval and spectrum based bug localization: better together , 2015, ESEC/SIGSOFT FSE.

[126]  Eric Lahtinen,et al.  Automatic error elimination by horizontal code transfer across multiple applications , 2015, PLDI.

[127]  Abhik Roychoudhury,et al.  DirectFix: Looking for Simple Program Repairs , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[128]  Abhik Roychoudhury,et al.  relifix: Automated Repair of Software Regressions , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[129]  Michael D. Ernst,et al.  Defects4J: a database of existing faults to enable controlled testing studies for Java programs , 2014, ISSTA 2014.

[130]  Martin Monperrus,et al.  A critical review of "automatic patch generation learned from human-written patches": essay on the problem statement and the evaluation of automatic software repair , 2014, ICSE.

[131]  Yuhua Qi,et al.  The strength of random search on automated program repair , 2014, ICSE.

[132]  James M. Bieman,et al.  An Assessment of the Quality of Automated Program Operator Repair , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[133]  Carlo A. Furia,et al.  Automatic Fixing of Programs with Contracts , 2015 .

[134]  Monperrus Martin,et al.  Mining software repair models for reasoning on the search space of automated program fixing , 2013, Empirical Software Engineering.

[135]  Westley Weimer,et al.  Leveraging program equivalence for adaptive program repair: Models and first results , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[136]  Yuhua Qi,et al.  Efficient Automated Program Repair through Fault-Recorded Testing Prioritization , 2013, 2013 IEEE International Conference on Software Maintenance.

[137]  Jaechang Nam,et al.  Automatic patch generation learned from human-written patches , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[138]  Dawei Qi,et al.  SemFix: Program repair via semantic analysis , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[139]  Westley Weimer,et al.  Automated repair of binary and assembly programs for cooperating embedded devices , 2013, ASPLOS '13.

[140]  Thomas Ball,et al.  Modular and verified automatic program repair , 2012, OOPSLA '12.

[141]  Claire Le Goues,et al.  Representations and operators for improving evolutionary software repair , 2012, GECCO '12.

[142]  Claire Le Goues,et al.  A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[143]  Charles Zhang,et al.  Axis: Automatically fixing atomicity violations through solving control constraints , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[144]  Claire Le Goues,et al.  GenProg: A Generic Method for Automatic Software Repair , 2012, IEEE Transactions on Software Engineering.

[145]  Shan Lu,et al.  Automated atomicity-violation fixing , 2011, PLDI '11.

[146]  Sarfraz Khurshid,et al.  Specification-Based Program Repair Using SAT , 2011, TACAS.

[147]  Bertrand Meyer,et al.  Code-based automated program fixing , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[148]  Westley Weimer,et al.  Automated program repair through the evolution of assembly code , 2010, ASE.

[149]  Claire Le Goues,et al.  Designing better fitness functions for automated program repair , 2010, GECCO '10.

[150]  Claire Le Goues,et al.  Automatic program repair with evolutionary computation , 2010, Commun. ACM.

[151]  W. Eric Wong,et al.  Using Mutation to Automatically Suggest Fixes for Faulty Programs , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[152]  Claire Le Goues,et al.  A genetic programming approach to automated software repair , 2009, GECCO.

[153]  Claire Le Goues,et al.  Automatically finding patches using genetic programming , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[154]  Sunghun Kim,et al.  Toward an understanding of bug fix patterns , 2009, Empirical Software Engineering.

[155]  Roderick Bloem,et al.  Repair of Boolean Programs with an Application to C , 2006, CAV.

[156]  Roderick Bloem,et al.  Program Repair as a Game , 2005, CAV.

[157]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.