Structurized grammar-based fuzz testing for programs with highly structured inputs

Fuzz testing, also known as fuzzing, has long been recognized as an effective technique to detect software vulnerabilities. Unfortunately, this approach is demonstrated noneffective when applied to test programs with highly structured inputs, such as interpreters and compilers. These programs usually process inputs in several stages as lexing and parsing, where the test input will be rejected if its structure does not obey the grammar. In this paper, we present a novel approach for fuzzing highly structured input programs. By disassembling the existing test cases into multiple grammatical fragments and inferring their grammar structures, we build a new series of test cases that can pass the validation and reach the previously unexplored places in the target program. We have implemented this approach in our general fuzzing framework BlendFuzz. Experiments have shown that BlendFuzz achieves higher code coverage compared with other blackbox fuzzing tools. BlendFuzz has also detected over two dozens of previously unreported vulnerabilities in real-world applications, with seven of them being considered high risky. Copyright © 2013 John Wiley & Sons, Ltd.

[1]  Barton P. Miller,et al.  Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services , 1995 .

[2]  Barton P. Miller,et al.  An empirical study of the reliability of UNIX utilities , 1990, Commun. ACM.

[3]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[4]  K. V. Hanford,et al.  Automatic Generation of Test Cases , 1970, IBM Syst. J..

[5]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[6]  Rupak Majumdar,et al.  Directed test generation using symbolic grammars , 2007, ASE.

[7]  David A. Wagner,et al.  Dynamic Test Generation to Find Integer Bugs in x86 Binary Linux Programs , 2009, USENIX Security Symposium.

[8]  Paul Walton Purdom,et al.  A sentence generator for testing parsers , 1972 .

[9]  Mikhail Posypkin,et al.  Survey of Compiler Testing Methods , 2005, Programming and Computer Software.

[10]  Peter Oehlert,et al.  Violating Assumptions with Fuzzing , 2005, IEEE Secur. Priv..

[11]  Adam Kiezun,et al.  Grammar-based whitebox fuzzing , 2008, PLDI '08.

[12]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[13]  Tevfik Bultan,et al.  Client and server verification for web services using interface grammars , 2008, TAV-WEB '08.

[14]  Stephen McCamant,et al.  Input generation via decomposition and re-stitching: finding bugs in Malware , 2010, CCS '10.

[15]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[16]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[17]  Guofei Gu,et al.  TaintScope: A Checksum-Aware Directed Fuzzing Tool for Automatic Software Vulnerability Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[18]  Xuejun Yang,et al.  Finding and understanding bugs in C compilers , 2011, PLDI '11.

[19]  W. M. McKeeman,et al.  Differential Testing for Software , 1998, Digit. Tech. J..

[20]  Richard L. Sauder,et al.  A general test data generator for COBOL , 1962, AIEE-IRE '62 (Spring).

[21]  Bruce McKenzie Generating Strings at Random from a Context Free Grammar , 1997 .