Static detection of C++ vtable escape vulnerabilities in binary code

Static binary code analysis is a longstanding technique used to find security defects in deployed proprietary software. The complexities of binary code compiled from object-oriented source languages (e.g. C++) has limited the utility of binary analysis to basic applications using simpler coding constructs, so vulnerabilities in object-oriented code remain undetected. In this paper, we present vtable escape bugs—a class of type confusion errors specific to C++ code present in real, deployed software including Adobe Reader, Microsoft Office, and the Windows subsystem DLLs. We developed automated binary code analyses able to statically detect vtable escape bugs by reconstructing high-level objects and analyzing the safety of their use. We implemented our analysis in our own general object code decompilation framework to demonstrate that classes of object-oriented vulnerabilities can be uncovered from compiled binaries. We successfully found vtable escape bugs in a collection of test samples that mimic publicly disclosed vulnerabilities in Adobe Reader and Microsoft Excel. With these new analyses, security analysts gain the ability to find common flaws introduced by applications compiled from C++.

[1]  Zhenkai Liang,et al.  BitBlaze: A New Approach to Computer Security via Binary Analysis , 2008, ICISS.

[2]  Herbert Bos,et al.  Howard: A Dynamic Excavator for Reverse Engineering Data Structures , 2011, NDSS.

[3]  David A. Wagner,et al.  This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. Detecting Format String Vulnerabilities with Type Qualifiers , 2001 .

[4]  David E. Evans,et al.  Static detection of dynamic memory errors , 1996, PLDI '96.

[5]  Thomas Dullien,et al.  REIL: A platform-independent intermediate representation of disassembled code for static code analysis , 2009 .

[6]  Dirk Grunwald,et al.  Reducing indirect function call overhead in C++ programs , 1994, POPL '94.

[7]  Barbara G. Ryder,et al.  Data-Flow-Based Virtual Function Resolution , 1996, SAS.

[8]  Gary McGraw,et al.  ITS4: a static vulnerability scanner for C and C++ code , 2000, Proceedings 16th Annual Computer Security Applications Conference (ACSAC'00).

[9]  Samuel T. King,et al.  Digging for Data Structures , 2008, OSDI.

[10]  David Evans,et al.  Improving Security Using Extensible Lightweight Static Analysis , 2002, IEEE Softw..

[11]  Barbara G. Ryder,et al.  Static Type Determination for C++ , 1994, C++ Conference.

[12]  Sean Heelan.  Vulnerability Detection Systems: Think Cyborg, Not Robot , 2011, IEEE Security & Privacy.

[13]  David F. Bacon,et al.  Fast static analysis of C++ virtual function calls , 1996, OOPSLA '96.

[14]  Dawson R. Engler,et al.  ARCHER: using symbolic, path-sensitive analysis to detect memory access errors , 2003, ESEC/FSE-11.

[15]  Catherine A. Meadows.  A procedure for verifying security against type confusion attacks , 2003, 16th IEEE Computer Security Foundations Workshop, 2003. Proceedings..

[16]  William R. Bush,et al.  A static analyzer for finding dynamic programming errors , 2000, Softw. Pract. Exp..

[17]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[18]  Abhinav Srivastava,et al.  Robust signatures for kernel data structures , 2009, CCS.

[19]  Todd M. Austin,et al.  Efficient detection of all pointer and array access errors , 1994, PLDI '94.

[20]  Murray Hill,et al.  Lint, a C Program Checker , 1978 .

[21]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[22]  Dinakar Dhurjati,et al.  SAFECode: enforcing alias analysis for weakly typed languages , 2005, PLDI '06.

[23]  David Evans,et al.  Statically Detecting Likely Buffer Overflow Vulnerabilities , 2001, USENIX Security Symposium.

[24]  Bjarne Stroustrup,et al.  The Design and Evolution of C , 1994 .