Technical Report 1112 Coverage Maximization Using Dynamic Taint Tracing

We present COMET, a system that automatically assembles a test suite for a C program to improve line coverage, and give initial results for a prototype implementation. COMET works dynamically, running the program under a variety of instrumentations in a feedback loop that adds new inputs to an initial corpus with each iteration. One instrumentation in particular is crucial to the success of this approach: dynamic taint tracing. Inputs are labeled as tainted at the byte level and all read/write pairs in the program are augmented to track the flow of taint between memory objects. This allows COMET to determine from which bytes of which inputs the variables in conditions derive, thereby dramatically narrowing the search over inputs necessary to expose new code. On a test set of 13 example programs, COMET improves upon the level of coverage reached in random testing by an average of 23% relative, takes only about twice the time, and requires a tiny fraction of the number of inputs to do so.

[1]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[2]  Yannis Smaragdakis,et al.  DSD-Crasher: A hybrid analysis tool for bug finding , 2006, TSEM.

[3]  Thomas A. Henzinger,et al.  SYNERGY: a new algorithm for property checking , 2006, SIGSOFT '06/FSE-14.

[4]  Wei Xu,et al.  Taint-Enhanced Policy Enforcement: A Practical Approach to Defeat a Wide Range of Attacks , 2006, USENIX Security Symposium.

[5]  Thomas Ball,et al.  Testing, abstraction, theorem proving: better together! , 2006, ISSTA '06.

[6]  Barton P. Miller,et al.  An empirical study of the robustness of MacOS applications using random testing , 2006, RT '06.

[7]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[8]  Dawson R. Engler,et al.  Execution Generated Test Cases: How to Make Systems Code Crash Itself , 2005, SPIN.

[9]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[10]  Anh Nguyen-Tuong,et al.  Automatically Hardening Web Applications Using Precise Tainting , 2005, SEC.

[11]  James Newsome,et al.  Dynamic Taint Analysis for Automatic Detection, Analysis, and SignatureGeneration of Exploits on Commodity Software , 2005, NDSS.

[12]  R. Sekar,et al.  Practical Dynamic Taint Analysis for Countering Input Validation Attacks on Web Applications , 2005 .

[13]  Thomas Ball,et al.  A Theory of Predicate-Complete Test Coverage and Generation , 2004, FMCO.

[14]  Richard Lippmann,et al.  Testing static analysis tools using exploitable buffer overflows from open source code , 2004, SIGSOFT '04/FSE-12.

[15]  Tal Garfinkel,et al.  Understanding data lifetime via whole system simulation , 2004 .

[16]  Olatunji Ruwase,et al.  A Practical Dynamic Buffer Overflow Detector , 2004, NDSS.

[17]  Andrew C. Myers,et al.  Language-based information-flow security , 2003, IEEE J. Sel. Areas Commun..

[18]  Misha Zitser Securing software : an evaluation of static source code analyzers , 2003 .

[19]  Gregg Rothermel,et al.  Test reuse in the spreadsheet paradigm , 2002, 13th International Symposium on Software Reliability Engineering, 2002. Proceedings..

[20]  George C. Necula,et al.  CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs , 2002, CC.

[21]  William H. Press,et al.  Numerical recipes in C , 2002 .

[22]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[23]  Parasuram Anantharam Programming ruby , 2001, SOEN.

[24]  Michael Hind,et al.  Pointer analysis: haven't we solved this problem yet? , 2001, PASTE '01.

[25]  Vtt Publications,et al.  A Functional Method for Assessing Protocol Implementation Security , 2001 .

[26]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[27]  Mary Lou Soffa,et al.  Automated test data generation using an iterative relaxation method , 1998, SIGSOFT '98/FSE-6.

[28]  Paul H. J. Kelly,et al.  Backwards-Compatible Bounds Checking for Arrays and Pointers in C Programs , 1997, AADEBUG.

[29]  Bogdan Korel,et al.  Automated test data generation for programs with procedures , 1996, ISSTA '96.

[30]  Bogdan Korel,et al.  The chaining approach for software test data generation , 1996, TSEM.

[31]  Paul C. Jorgensen,et al.  Software Testing: A Craftsman's Approach , 1995 .

[32]  Larry Wall,et al.  Programming Perl , 1991 .

[33]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..