Directed Greybox Fuzzing

Existing Greybox Fuzzers (GF) cannot be effectively directed, for instance, towards problematic changes or patches, towards critical system calls or dangerous locations, or towards functions in the stack-trace of a reported vulnerability that we wish to reproduce. In this paper, we introduce Directed Greybox Fuzzing (DGF) which generates inputs with the objective of reaching a given set of target program locations efficiently. We develop and evaluate a simulated annealing-based power schedule that gradually assigns more energy to seeds that are closer to the target locations while reducing energy for seeds that are further away. Experiments with our implementation AFLGo demonstrate that DGF outperforms both directed symbolic-execution-based whitebox fuzzing and undirected greybox fuzzing. We show applications of DGF to patch testing and crash reproduction, and discuss the integration of AFLGo into Google's continuous fuzzing platform OSS-Fuzz. Due to its directedness, AFLGo could find 39 bugs in several well-fuzzed, security-critical projects like LibXML2. 17 CVEs were assigned.

[1]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[2]  Erik van der Kouwe,et al.  TypeSan: Practical Type Confusion Detection , 2016, CCS.

[3]  Sarfraz Khurshid,et al.  Directed incremental symbolic execution , 2011, PLDI '11.

[4]  Christopher Krügel,et al.  Driller: Augmenting Fuzzing Through Selective Symbolic Execution , 2016, NDSS.

[5]  Patrice Godefroid,et al.  SAGE: Whitebox Fuzzing for Security Testing , 2012, ACM Queue.

[6]  Alessandro Orso,et al.  Test-Suite Augmentation for Evolving Software , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[7]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[8]  Abhik Roychoudhury,et al.  Model-based whitebox fuzzing for program binaries , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[9]  Derek Bruening,et al.  AddressSanitizer: A Fast Address Sanity Checker , 2012, USENIX Annual Technical Conference.

[10]  Abhik Roychoudhury,et al.  Hercules: Reproducing Crashes in Real-World Application Binaries , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[11]  Ryan Cunningham,et al.  Automated Vulnerability Analysis: Leveraging Control Flow for Evolutionary Input Crafting , 2007, Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007).

[12]  Barton P. Miller,et al.  An empirical study of the reliability of UNIX utilities , 1990, Commun. ACM.

[13]  Martin C. Rinard,et al.  Taint-based directed whitebox fuzzing , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[14]  Guofei Gu,et al.  TaintScope: A Checksum-Aware Directed Fuzzing Tool for Automatic Software Vulnerability Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[15]  Kurt Mehlhorn,et al.  Data Structures and Algorithms 1: Sorting and Searching , 2011, EATCS Monographs on Theoretical Computer Science.

[16]  Andreas Zeller,et al.  Detecting information flow by mutating input data , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[17]  Lionel C. Briand,et al.  A Hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering , 2014, Softw. Test. Verification Reliab..

[18]  Michael Hicks,et al.  Directed Symbolic Execution , 2011, SAS.

[19]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[20]  Prof. Dr. Kurt Mehlhorn,et al.  Data Structures and Algorithms 1 , 1984, EATCS.

[21]  Bruno C. d. S. Oliveira,et al.  Partition-based regression verification , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[22]  Alessandro Orso,et al.  BugRedux: Reproducing field failures for in-house debugging , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[23]  Mark Harman,et al.  Strong higher order mutation-based test data generation , 2011, ESEC/FSE '11.

[24]  Peter Müller,et al.  Guiding Dynamic Symbolic Execution toward Unverified Program Executions , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[25]  Myra B. Cohen,et al.  Directed test suite augmentation: techniques and tradeoffs , 2010, FSE '10.

[26]  Brian S. Pak,et al.  Hybrid Fuzz Testing: Discovering Software Bugs via Fuzzing and Symbolic Execution , 2012 .

[27]  George Candea,et al.  S2E: a platform for in-vivo multi-path analysis of software systems , 2011, ASPLOS XVI.

[28]  Andreas Zeller,et al.  Reconstructing Core Dumps , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[29]  Heng Yin,et al.  DECAF: A Platform-Neutral Whole-System Dynamic Binary Analysis Platform , 2017, IEEE Transactions on Software Engineering.

[30]  Herbert Bos,et al.  Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations , 2013, USENIX Security Symposium.

[31]  Zhenkai Liang,et al.  Test generation to expose changes in evolving programs , 2010, ASE '10.

[32]  Konstantin Serebryany,et al.  MemorySanitizer: Fast detector of uninitialized memory use in C++ , 2015, 2015 IEEE/ACM International Symposium on Code Generation and Optimization (CGO).

[33]  Abhik Roychoudhury,et al.  Coverage-Based Greybox Fuzzing as Markov Chain , 2016, IEEE Transactions on Software Engineering.

[34]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[35]  Soumya Paul,et al.  A Probabilistic Analysis of the Efficiency of Automated Software Testing , 2016, IEEE Transactions on Software Engineering.

[36]  Cristian Cadar,et al.  KATCH: high-coverage testing of software patches , 2013, ESEC/FSE 2013.

[37]  A. Vargha,et al.  A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong , 2000 .

[38]  Bruno C. d. S. Oliveira,et al.  Regression tests to expose change interaction errors , 2013, ESEC/FSE 2013.

[39]  Herbert Bos,et al.  VUzzer: Application-aware Evolutionary Fuzzing , 2017, NDSS.

[40]  Matthew B. Dwyer,et al.  On the Techniques We Create, the Tools We Build, and Their Misalignments: A Study of KLEE , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).