FUDGE: fuzz driver generation at scale

At Google we have found tens of thousands of security and robustness bugs by fuzzing C and C++ libraries. To fuzz a library, a fuzzer requires a fuzz driver—which exercises some library code—to which it can pass inputs. Unfortunately, writing fuzz drivers remains a primarily manual exercise, a major hindrance to the widespread adoption of fuzzing. In this paper, we address this major hindrance by introducing the Fudge system for automated fuzz driver generation. Fudge automatically generates fuzz driver candidates for libraries based on existing client code. We have used Fudge to generate thousands of new drivers for a wide variety of libraries. Each generated driver includes a synthesized C/C++ program and a corresponding build script, and is automatically analyzed for quality. Developers have integrated over 200 of these generated drivers into continuous fuzzing services and have committed to address reported security bugs. Further, several of these fuzz drivers have been upstreamed to open source projects and integrated into the OSS-Fuzz fuzzing infrastructure. Running these fuzz drivers has resulted in over 150 bug fixes, including the elimination of numerous exploitable security vulnerabilities.

[1]  Marco Tulio Valente,et al.  Documenting APIs with examples: Lessons learned with the APIMiner platform , 2013, 2013 20th Working Conference on Reverse Engineering (WCRE).

[2]  Gonçalo R. Abecasis,et al.  The Sequence Alignment/Map format and SAMtools , 2009, Bioinform..

[3]  Charles A. Sutton,et al.  Parameter-free probabilistic API mining across GitHub , 2015, SIGSOFT FSE.

[4]  Derek Bruening,et al.  AddressSanitizer: A Fast Address Sanity Checker , 2012, USENIX Annual Technical Conference.

[5]  Michael D. Ernst,et al.  Finding the needles in the haystack: Generating legal test inputs for object-oriented programs , 2006 .

[6]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[7]  Craig Chambers,et al.  FlumeJava: easy, efficient data-parallel pipelines , 2010, PLDI '10.

[8]  Gabriele Bavota,et al.  How Can I Use This Method? , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[9]  Andreas Zeller,et al.  Carving Parameterized Unit Tests , 2018, 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion).

[10]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[11]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[12]  Jian Pei,et al.  MAPO: Mining and Recommending API Usage Patterns , 2009, ECOOP.

[13]  Patrice Godefroid,et al.  Micro execution , 2014, ICSE.

[14]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[15]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[16]  Michael D. Ernst,et al.  Combined static and dynamic automated test generation , 2011, ISSTA '11.

[17]  Frank Tip,et al.  A survey of program slicing techniques , 1994, J. Program. Lang..

[18]  Westley Weimer,et al.  Synthesizing API usage examples , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[19]  Seung-won Hwang,et al.  Enriching Documents with Examples: A Corpus Mining Approach , 2013, TOIS.

[20]  Matthew B. Dwyer,et al.  Carving and Replaying Differential Unit Test Cases from System Test Cases , 2009, IEEE Transactions on Software Engineering.

[21]  R. Smith,et al.  An Overview of the Tesseract OCR Engine , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[22]  Kai Chen,et al.  Mining succinct and high-coverage API usage patterns from source code , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[23]  Hyrum K. Wright,et al.  Large-Scale Automated Refactoring Using ClangMR , 2013, 2013 IEEE International Conference on Software Maintenance.

[24]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[25]  Houari A. Sahraoui,et al.  Mining Multi-level API Usage Patterns , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[26]  Kathryn T. Stolee,et al.  How developers search for code: a case study , 2015, ESEC/SIGSOFT FSE.

[27]  Lu Zhang,et al.  Mining API Usage Examples from Test Code , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[28]  Charles A. Sutton,et al.  Summarizing Software API Usage Examples Using Clustering Techniques , 2018, FASE.

[29]  Choongwoo Han,et al.  Fuzzing: Art, Science, and Engineering , 2018, ArXiv.

[30]  Tao Xie,et al.  Random unit-test generation with MUT-aware sequence recommendation , 2010, ASE '10.

[31]  Josh Levenberg,et al.  Why Google stores billions of lines of code in a single repository , 2016, Commun. ACM.