Beyond Tests: Program Vulnerability Repair via Crash Constraint Extraction

Automated program repair is an emerging technology which seeks to automatically rectify program errors and vulnerabilities. Repair techniques are driven by a correctness criterion which is often in the form of a test-suite. Such test-based repair may produce over-fitting patches, where the patches produced fail on tests outside the test-suite driving the repair. In this work, we present a repair method which fixes program vulnerabilities without the need for a voluminous test-suite. Given a vulnerability as evidenced by an exploit, the technique extracts a constraint representing the vulnerability with the help of sanitizers. The extracted constraint serves as a proof obligation which our synthesized patch should satisfy. The proof obligation is met by propagating the extracted constraint to locations which are deemed to be "suitable" fix locations. An implementation of our approach (ExtractFix) on top of the KLEE symbolic execution engine shows its efficacy in fixing a wide range of vulnerabilities taken from ManyBugs benchmark, real-world CVEs and Google’s Open-source-systems OSS Fuzz framework. We believe that our work presents a way forward for the overfitting problem in program repair, by generalizing observable hazards/vulnerabilities (as constraint) from a single failing test or exploit.

[1]  Mark N. Wegman,et al.  Efficiently computing static single assignment form and the control dependence graph , 1991, TOPL.

[2]  Frank Tip,et al.  A survey of program slicing techniques , 1994, J. Program. Lang..

[3]  James R. Larus,et al.  The use of program profiling for software maintenance with applications to the year 2000 problem , 1997, ESEC '97/FSE-5.

[4]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[5]  Sharad Malik,et al.  On Solving the Partial MAX-SAT Problem , 2006, SAT.

[6]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[7]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[8]  Manu Sridharan,et al.  Snugglebug: a powerful approach to weakest preconditions , 2009, PLDI '09.

[9]  Sumit Gulwani,et al.  Oracle-guided component-based program synthesis , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[10]  Ding Yuan,et al.  How do fixes become bugs? , 2011, ESEC/FSE '11.

[11]  David Brumley,et al.  Efficient Directionless Weakest Preconditions , 2011 .

[12]  Wei Zhang,et al.  Automated Concurrency-Bug Fixing , 2012, OSDI.

[13]  Derek Bruening,et al.  AddressSanitizer: A Fast Address Sanity Checker , 2012, USENIX Annual Technical Conference.

[14]  Claire Le Goues,et al.  GenProg: A Generic Method for Automatic Software Repair , 2012, IEEE Transactions on Software Engineering.

[15]  Dawei Qi,et al.  SemFix: Program repair via semantic analysis , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[16]  Baowen Xu,et al.  A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization , 2013, TSEM.

[17]  Chen Liu,et al.  R2Fix: Automatically Generating Bug Fixes from Bug Reports , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[18]  Yuhua Qi,et al.  The strength of random search on automated program repair , 2014, ICSE.

[19]  Abhik Roychoudhury,et al.  DirectFix: Looking for Simple Program Repairs , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[20]  Lu Zhang,et al.  Safe Memory-Leak Fixing for C Programs , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[21]  Yuriy Brun,et al.  Is the cure worse than the disease? overfitting in automated program repair , 2015, ESEC/SIGSOFT FSE.

[22]  Yuriy Brun,et al.  The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs , 2015, IEEE Transactions on Software Engineering.

[23]  Yuriy Brun,et al.  Repairing Programs with Semantic Code Search (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[24]  Abhik Roychoudhury,et al.  Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[25]  Roland H. C. Yap,et al.  Heap bounds protection with low fat pointers , 2016, CC.

[26]  Fan Long,et al.  Automatic patch generation by learning correct code , 2016, POPL.

[27]  Yan Cai,et al.  Fixing Deadlocks via Lock Pre-Acquisitions , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[28]  David Lo,et al.  History Driven Program Repair , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[29]  David Lie,et al.  Talos: Neutralizing Vulnerabilities with Security Workarounds for Rapid Response , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[30]  Roland H. C. Yap,et al.  Stack Bounds Protection with Low Fat Pointers , 2017, NDSS.

[31]  Lu Zhang,et al.  Can big data bring a breakthrough for software automation? , 2018, Science China Information Sciences.

[32]  Claire Le Goues,et al.  JFIX: semantics-based repair of Java programs via symbolic PathFinder , 2017, ISSTA.

[33]  Qi Xin,et al.  Identifying test-suite-overfitted patches through test case generation , 2017, ISSTA.

[34]  David Lo,et al.  Overfitting in semantics-based automated program repair , 2018, Empirical Software Engineering.

[35]  Jiachen Zhang,et al.  Precise Condition Synthesis for Program Repair , 2016, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[36]  Martin Monperrus,et al.  Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs , 2018, IEEE Transactions on Software Engineering.

[37]  Alexey Zhikhartsev,et al.  Better test cases for better automated program repair , 2017, ESEC/SIGSOFT FSE.

[38]  Hiroaki Yoshida,et al.  Elixir: Effective object-oriented program repair , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[39]  Gang Huang,et al.  Identifying Patch Correctness in Test-Based Program Repair , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[40]  Alberto Griggio,et al.  Symbolic execution with existential second-order constraints , 2018, ESEC/SIGSOFT FSE.

[41]  Hongyu Zhang,et al.  Shaping program repair space with existing patches and similar code , 2018, ISSTA.

[42]  Claire Le Goues,et al.  Static Automated Program Repair for Heap Properties , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[43]  Lars Grunske,et al.  Semantic Program Repair Using a Reference Implementation , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[44]  Hakjoo Oh,et al.  MemFix: static analysis-based repair of memory deallocation errors for C , 2018, ESEC/SIGSOFT FSE.

[45]  Xiang Gao,et al.  Test-Equivalence Analysis for Automatic Patch Generation , 2018, ACM Trans. Softw. Eng. Methodol..

[46]  Ming Wen,et al.  Context-Aware Patch Generation for Better Automated Program Repair , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[47]  Daniela Micucci,et al.  Automatic Software Repair: A Survey , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[48]  Trent Jaeger,et al.  Using Safety Properties to Generate Vulnerability Patches , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[49]  Hua Yan,et al.  VFix: Value-Flow-Guided Precise Program Repair for Null Pointer Dereferences , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[50]  Hiroaki Yoshida,et al.  Phoenix: automated data-driven synthesis of repairs for static analysis violations , 2019, ESEC/SIGSOFT FSE.

[51]  Wei Li,et al.  DeepFL: integrating multiple fault diagnosis dimensions for deep fault localization , 2019, ISSTA.

[52]  Yingfei Xiong,et al.  Automated program repair: a step towards software automation , 2019, Science China Information Sciences.

[53]  Jacques Klein,et al.  iFixR: bug report driven program repair , 2019, ESEC/SIGSOFT FSE.

[54]  Jacques Klein,et al.  You Cannot Fix What You Cannot Find! An Investigation of Fault Localization Bias in Benchmarking Automated Program Repair Systems , 2018, 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST).

[55]  Claire Le Goues,et al.  Automated program repair , 2019, Commun. ACM.

[56]  Xiang Gao,et al.  Crash-avoiding program repair , 2019, ISSTA.

[57]  Abhik Roychoudhury,et al.  Interactive Patch Generation and Suggestion , 2020, ICSE.

[58]  Monperrus Martin Automatic Software Repair: a Bibliography , 2020 .