Reconstructing program memory state from multi-gigabyte instruction traces to support interactive analysis

Exploitability analysis is the process of attempting to determine if a vulnerability in a program is exploitable. Fuzzing is a popular method of finding such vulnerabilities, in which a program is subjected to millions of generated program inputs until it crashes. Each program crash indicates a potential vulnerability that needs to be prioritized according to its potential for exploitation. The highest priority vulnerabilities need to be investigated by a security analyst by re-executing the program with the input that caused the crash while recording a trace of all executed assembly instructions and then performing analysis on the resulting trace. Recreating the entire memory state of the program at the time of the crash, or at any other point in the trace, is very important for helping the analyst build an understanding of the conditions that led to the crash. Unfortunately, tracing even a small program can create multimillion line trace files from which reconstructing memory state is a computationally intensive process and virtually impossible to do manually. In this paper we present an analysis of the problem of memory state reconstruction from very large execution traces. We report on a novel approach for reconstructing the entire memory state of a program from an execution trace that allows near realtime queries on the state of memory at any point in a program's execution trace. Finally we benchmark our approach showing storage and performance results in line with our theoretical calculations and demonstrate memory state query response times of less than 200ms for trace files up to 60 million lines.

[1]  Steven G. Parker,et al.  Interactive Visualization for Memory Reference Traces , 2008, Comput. Graph. Forum.

[2]  Bongshin Lee,et al.  AllocRay: memory allocation visualization for unmanaged languages , 2010, SOFTVIS '10.

[3]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[4]  Andrew Warfield,et al.  Execution mining , 2012, VEE '12.

[5]  Zhenkai Liang,et al.  BitBlaze: A New Approach to Computer Security via Binary Analysis , 2008, ICISS.

[6]  David Brumley,et al.  BAP: A Binary Analysis Platform , 2011, CAV.

[7]  Éric Tanter,et al.  Summarized Trace Indexing and Querying for Scalable Back-in-Time Debugging , 2011, ECOOP.

[8]  Pedram Amini,et al.  Fuzzing: Brute Force Vulnerability Discovery , 2007 .

[9]  Tal Garfinkel,et al.  Understanding data lifetime via whole system simulation , 2004 .

[10]  Sara L. Su,et al.  Heapviz: Interactive heap visualization for program understanding and debugging , 2013, Inf. Vis..

[11]  Sara L. Su,et al.  Heapviz: interactive heap visualization for program understanding and debugging , 2010, SOFTVIS '10.

[12]  Margaret-Anne D. Storey,et al.  ATLANTIS - Assembly Trace Analysis Environment , 2012, 2012 19th Working Conference on Reverse Engineering.

[13]  Wim De Pauw,et al.  Zinsight: a visual and analytic environment for exploring large event traces , 2010, SOFTVIS '10.

[14]  Jürgen Döllner,et al.  Understanding complex multithreaded software systems by using trace visualization , 2010, SOFTVIS '10.

[15]  Mireille Ducassé,et al.  Using events to debug Java programs backwards in time , 2003, OOPSLA '03.

[16]  Éric Tanter,et al.  Back to the Future: Omniscient Debugging , 2009, IEEE Software.

[17]  Christoph Treude,et al.  An Exploratory Study of Software Reverse Engineering in a Security Context , 2011, 2011 18th Working Conference on Reverse Engineering.