Fixing Rust Compilation Errors using LLMs

The Rust programming language, with its safety guarantees, has established itself as a viable choice for low-level systems programming language over the traditional, unsafe alternatives like C/C++. These guarantees come from a strong ownership-based type system, as well as primitive support for features like closures, pattern matching, etc., that make the code more concise and amenable to reasoning. These unique Rust features also pose a steep learning curve for programmers. This paper presents a tool called RustAssistant that leverages the emergent capabilities of Large Language Models (LLMs) to automatically suggest fixes for Rust compilation errors. RustAssistant uses a careful combination of prompting techniques as well as iteration with an LLM to deliver high accuracy of fixes. RustAssistant is able to achieve an impressive peak accuracy of roughly 74% on real-world compilation errors in popular open-source Rust repositories. We plan to release our dataset of Rust compilation errors to enable further research.

[1]  Eric Michael Smith,et al.  Llama 2: Open Foundation and Fine-Tuned Chat Models , 2023, ArXiv.

[2]  Andrew M. Dai,et al.  PaLM 2 Technical Report , 2023, ArXiv.

[3]  Ali Mesbah,et al.  Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning , 2023, 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE).

[4]  Lingming Zhang,et al.  Automated Program Repair in the Era of Large Pre-trained Language Models , 2023, 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE).

[5]  Zhengzi Xu,et al.  A Survey on Automated Program Repair Techniques , 2023, ArXiv.

[6]  Kevin Jesse,et al.  Large Language Models and Simple, Stupid Bugs , 2023, 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR).

[7]  Lingming Zhang,et al.  Revisiting the Plastic Surgery Hypothesis via Large Language Models , 2023, ArXiv.

[8]  Naman Goyal,et al.  LLaMA: Open and Efficient Foundation Language Models , 2023, ArXiv.

[9]  X. Zhang,et al.  KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair , 2023, 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE).

[10]  Lingming Zhang,et al.  Practical Program Repair in the Era of Large Pre-trained Language Models , 2022, ArXiv.

[11]  D. Lo,et al.  TransplantFix: Graph Differencing-based Code Transplantation for Automated Program Repair , 2022, ASE.

[12]  He He,et al.  Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought , 2022, ICLR.

[13]  Sumit Gulwani,et al.  Repair Is Nearly Generation: Multilingual Program Repair with LLMs , 2022, AAAI.

[14]  Shin Hwei Tan,et al.  Automated Repair of Programs from Large Language Models , 2022, 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE).

[15]  Julian Aron Prenner,et al.  Can OpenAI's Codex Fix Bugs?: An evaluation on QuixBugs , 2022, 2022 IEEE/ACM International Workshop on Automated Program Repair (APR).

[16]  Ali Mesbah,et al.  Katana: Dual Slicing Based Context for Learning Bug Fixes , 2022, ACM Trans. Softw. Eng. Methodol..

[17]  Sida I. Wang,et al.  InCoder: A Generative Model for Code Infilling and Synthesis , 2022, ICLR.

[18]  Ryan J. Lowe,et al.  Training language models to follow instructions with human feedback , 2022, NeurIPS.

[19]  D. Poshyvanyk,et al.  Can We Automatically Fix Bugs by Learning Edit Operations? , 2022, 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER).

[20]  Sufyan bin Uzayr GitHub , 2022, Mastering Git.

[21]  Brendan Dolan-Gavitt,et al.  Examining Zero-Shot Vulnerability Repair with Large Language Models , 2021, 2023 IEEE Symposium on Security and Privacy (SP).

[22]  David Bieber,et al.  Show Your Work: Scratchpads for Intermediate Computation with Language Models , 2021, ArXiv.

[23]  Yue Wang,et al.  CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation , 2021, EMNLP.

[24]  Ilya Sergey,et al.  Hippodrome: Data Race Repair Using Static Analysis Summaries , 2021, ACM Trans. Softw. Eng. Methodol..

[25]  Lu Zhang,et al.  A syntax-guided edit decoder for neural program repair , 2021, ESEC/SIGSOFT FSE.

[26]  Monperrus Martin,et al.  Neural Program Repair with Execution-based Backpropagation , 2021, 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE).

[27]  Noah Rose Ledesma,et al.  SynShine: Improved Fixing of Syntax Errors , 2021, IEEE Transactions on Software Engineering.

[28]  Thibaud Lutellier,et al.  CURE: Code-Aware Neural Machine Translation for Automatic Program Repair , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).

[29]  Shaohua Wang,et al.  DLFix: Context-based Code Transformation Learning for Automated Program Repair , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE).

[30]  Shangwen Wang The Dataset of ICSE'20 paper titled "On the Efficiency of Test Suite based Program Repair" , 2020 .

[31]  Claire Le Goues,et al.  Automated program repair , 2019, Commun. ACM.

[32]  Trent Jaeger,et al.  Using Safety Properties to Generate Vulnerability Patches , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[33]  Gabriele Bavota,et al.  An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation , 2018, ACM Trans. Softw. Eng. Methodol..

[34]  Tegawendé F. Bissyandé,et al.  AVATAR : Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations , 2018, SANER.

[35]  Martin Monperrus,et al.  Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs , 2018, IEEE Transactions on Software Engineering.

[36]  Xiang Gao,et al.  Test-Equivalence Analysis for Automatic Patch Generation , 2018, ACM Trans. Softw. Eng. Methodol..

[37]  Hongyu Zhang,et al.  Shaping program repair space with existing patches and similar code , 2018, ISSTA.

[38]  Ming Wen,et al.  Context-Aware Patch Generation for Better Automated Program Repair , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[39]  Xiang Gao,et al.  Repairing Crashes in Android Apps , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[40]  Claire Le Goues,et al.  Static Automated Program Repair for Heap Properties , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[41]  Wolfgang Banzhaf,et al.  ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programming , 2017, IEEE Transactions on Software Engineering.

[42]  Rahul Gupta,et al.  DeepFix: Fixing Common C Language Errors by Deep Learning , 2017, AAAI.

[43]  Martin White,et al.  Deep learning code fragments for code clone detection , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[44]  Abhik Roychoudhury,et al.  Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[45]  Abhik Roychoudhury,et al.  relifix: Automated Repair of Software Regressions , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[46]  Michael D. Ernst,et al.  Defects4J: a database of existing faults to enable controlled testing studies for Java programs , 2014, ISSTA 2014.

[47]  Dawei Qi,et al.  SemFix: Program repair via semantic analysis , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[48]  亀井 靖高,et al.  Stack Overflow , 2012 .

[49]  44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022 , 2022, ICSE.

[50]  ESEC/FSE '21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece, August 23-28, 2021 , 2021, ESEC/SIGSOFT FSE.

[51]  Ramesh Karri,et al.  Can OpenAI Codex and Other Large Language Models Help Us Fix Security Bugs? , 2021, ArXiv.

[52]  Monperrus Martin Automatic Software Repair: a Bibliography , 2020 .

[53]  知秋 Microsoft:微软“变脸” , 2006 .