EXE: A system for automatically generating inputs of death using symbolic execution

Systems code defines an error-prone execution state space built from deeply nested conditionals and function call chains, massive amounts of code, and enthusiastic use of casting and pointer operations. Such code is hard to test and difficult to inspect, yet a single error can crash a machine or form the basis of a security breach. This paper presents EXE, a system designed to automatically find bugs in such code using symbolic execution. At a high level, rather than running the code on manually-constructed concrete input, EXE instead runs it on symbolic input that is initially allowed to be “anything.” As input (and derived) values are observed through conditional statements and other checks, symbolic constraints are incrementally added to those values. EXE then generates concrete test cases by solving these symbolic constraints for concrete values with bit-level precision. EXE has several novel features. First, it implements a complete, precise symbolic pointer theory that correctly handles both pointer arithmetic expressions and reads and writes to memory locations referenced by pointers with symbolic values. Second, it handles all of the C language with bit-level precision. Third, EXE greatly amplifies the effect of running a single code path since it uses a powerful constraint solver to reason about all possible values that the path could be run with, rather than a single set of concrete values from an individual test case. EXE has been successfully applied to applications ranging from running the Linux kernel symbolically in order to find numerous security holes in the ext2, ext3, and JFS file systems [26] to detecting invalid memory reads and writes in a DHCPD server implementation to finding buffer overflow attacks in the BSD and Linux packet filter implementations.

[1]  Karl N. Levitt,et al.  SELECT—a formal system for testing and debugging programs by symbolic execution , 1975 .

[2]  Greg Nelson,et al.  Simplification by Cooperating Decision Procedures , 1979, TOPL.

[3]  Robert O. Hastings,et al.  Fast detection of memory leaks and access errors , 1991 .

[4]  Bogdan Korel,et al.  The chaining approach for software test data generation , 1996, TSEM.

[5]  Patrice Godefroid,et al.  Model checking for programming languages using VeriSoft , 1997, POPL '97.

[6]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[7]  Mary Lou Soffa,et al.  Automated test data generation using an iterative relaxation method , 1998, SIGSOFT '98/FSE-6.

[8]  Arnaud Gotlieb,et al.  Automatic test data generation using constraint solving techniques , 1998, ISSTA '98.

[9]  David A. Wagner,et al.  A First Step Towards Automated Detection of Buffer Overrun Vulnerabilities , 2000, NDSS.

[10]  Matthew B. Dwyer,et al.  Bandera: extracting finite-state models from Java source code , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[11]  William R. Bush,et al.  A static analyzer for finding dynamic programming errors , 2000, Softw. Pract. Exp..

[12]  Junfeng Yang,et al.  An empirical study of operating systems errors , 2001, SOSP.

[13]  Sriram K. Rajamani,et al.  Automatically validating temporal safety properties of interfaces , 2001, SPIN '01.

[14]  Alexander Aiken,et al.  Flow-sensitive type qualifiers , 2002, PLDI '02.

[15]  George C. Necula,et al.  CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs , 2002, CC.

[16]  Sorin Lerner,et al.  ESP: path-sensitive program verification in polynomial time , 2002, PLDI '02.

[17]  Sarfraz Khurshid,et al.  Generalized Symbolic Execution for Model Checking and Testing , 2003, TACAS.

[18]  Todd M. Austin,et al.  High Coverage Detection of Input-Related Security Faults , 2003, USENIX Security Symposium.

[19]  E. Clarke,et al.  Hardware verification using ANSI-C programs as a reference , 2003, Proceedings of the ASP-DAC Asia and South Pacific Design Automation Conference, 2003..

[20]  Thomas Ball,et al.  A Theory of Predicate-Complete Test Coverage and Generation , 2004, FMCO.

[21]  Olatunji Ruwase,et al.  A Practical Dynamic Buffer Overflow Detector , 2004, NDSS.

[22]  Sergey Berezin,et al.  CVC Lite: A New Implementation of the Cooperating Validity Checker Category B , 2004, CAV.

[23]  Dawson R. Engler,et al.  Model Checking Large Network Protocol Implementations , 2004, NSDI.

[24]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[25]  Dawson R. Engler,et al.  Execution Generated Test Cases: How to Make Systems Code Crash Itself , 2005, SPIN.

[26]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[27]  Alexander Aiken,et al.  Scalable error detection using boolean satisfiability , 2005, POPL '05.

[28]  Brian N. Bershad,et al.  Recovering device drivers , 2004, TOCS.

[29]  Junfeng Yang,et al.  Automatically generating malicious disks using symbolic execution , 2006, 2006 IEEE Symposium on Security and Privacy (S&P'06).