QuickFuzz testing for fun and profit

Abstract Fuzzing is a popular technique to find flaws in programs using invalid or erroneous inputs but not without its drawbacks. At one hand, mutational fuzzers require a set of valid inputs as a starting point, in which modifications are then introduced. On the other hand, generational fuzzing allows to synthesize somehow valid inputs according to a specification. Unfortunately, this requires to have a deep knowledge of the file formats under test to write specifications of them to guide the test case generation process. In this paper we introduce an extended and improved version of QuickFuzz, a tool written in Haskell designed for testing unexpected inputs of common file formats on third-party software, taking advantage of off-the-self well known fuzzers. Unlike other generational fuzzers, QuickFuzz does not require to write specifications for the file formats in question since it relies on existing file-format-handling libraries available on the Haskell code repository. It supports almost 40 different complex file-types including images, documents, source code and digital certificates. In particular, we found QuickFuzz useful enough to discover many previously unknown vulnerabilities on real-world implementations of web browsers and image processing libraries among others.

[1]  David Brumley,et al.  Unleashing Mayhem on Binary Code , 2012, 2012 IEEE Symposium on Security and Privacy.

[2]  Derek Bruening,et al.  AddressSanitizer: A Fast Address Sanity Checker , 2012, USENIX Annual Technical Conference.

[3]  Barton P. Miller,et al.  An empirical study of the reliability of UNIX utilities , 1990, Commun. ACM.

[4]  Simon L. Peyton Jones,et al.  Template meta-programming for Haskell , 2002, Haskell '02.

[5]  Pedram Amini,et al.  Fuzzing: Brute Force Vulnerability Discovery , 2007 .

[6]  Emil Axelsson,et al.  Combining deep and shallow embedding of domain-specific languages , 2015, Comput. Lang. Syst. Struct..

[7]  Paul Hudak,et al.  Modular domain specific languages and tools , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[8]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[9]  John Hughes,et al.  Testing Erlang data types with quviq quickcheck , 2008, ERLANG '08.

[10]  Abhik Roychoudhury,et al.  Coverage-Based Greybox Fuzzing as Markov Chain , 2017, IEEE Trans. Software Eng..

[11]  Koen Claessen,et al.  Testing monadic code with QuickCheck , 2002, ACM SIGPLAN Notices.

[12]  Zachary N. J. Peterson,et al.  Analysis of Mutation and Generation-Based Fuzzing , 2007 .

[13]  Pablo Buiras,et al.  QuickFuzz: an automatic random fuzzer for common file formats , 2016, Haskell.

[14]  Nicholas Nethercote,et al.  Valgrind: a framework for heavyweight dynamic binary instrumentation , 2007, PLDI '07.

[15]  Simon Marlow,et al.  Haskell 2010 Language Report , 2010 .

[16]  Alexander Aiken,et al.  Synthesizing program input grammars , 2016, PLDI.

[17]  P. H. Testing an Optimising Compiler by Generating Random Lambda Terms , 2012 .

[18]  Burkhart Wolff,et al.  Automatic and efficient simulation of operation contracts , 2010, GPCE '10.

[19]  Andreas Zeller,et al.  Mining Input Grammars with AUTOGRAM , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[20]  Koen Claessen,et al.  Generating constrained random data with uniform distribution , 2014, Journal of Functional Programming.

[21]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[22]  Adam Kiezun,et al.  Grammar-based whitebox fuzzing , 2008, PLDI '08.

[23]  Conor McBride,et al.  Applicative programming with effects , 2008, J. Funct. Program..