Fast and practical instruction-set randomization for commodity systems

Instruction-set randomization (ISR) is a technique based on randomizing the "language" understood by a system to protect it from code-injection attacks. Such attacks were used by many computer worms in the past, but still pose a threat as it was confirmed by the recent Conficker worm outbreak, and the latest exploits targeting some of Adobe's most popular products. This paper presents a fast and practical implementation of ISR that can be applied on currently deployed software. Our solution builds on a binary instrumentation tool to provide an ISR-enabled execution environment entirely in software. Applications are randomized using a simple XOR function and a 16-bit key that is randomly generated every time an application is launched. Shared libraries can be also randomized using separate keys, and their randomized versions can be used by all applications running under ISR. Moreover, we introduce a key management system to keep track of the keys used in the system. To the best of our knowledge we are the first to apply ISR on truly shared libraries. Finally, we evaluate our implementation using real applications including the Apache web server, and the MySQL database server. For the first, we show that our implementation has negligible overhead (less than 1%) for static HTML loads, while the overhead when running MySQL can be as low as 75%. We see that our system can be used with little cost with I/O intensive network applications, while it can also be a good candidate for deployment with CPU intensive applications, in scenarios where security outweighs performance.

[1]  Daniel C. DuVarney,et al.  Address Obfuscation: An Efficient Approach to Combat a Broad Range of Memory Error Exploits , 2003, USENIX Security Symposium.

[2]  Hovav Shacham,et al.  On the effectiveness of address-space randomization , 2004, CCS '04.

[3]  Jonathan D. Pincus,et al.  Beyond stack smashing: recent advances in exploiting buffer overruns , 2004, IEEE Security & Privacy Magazine.

[4]  George C. Necula,et al.  CCured: type-safe retrofitting of legacy software , 2005, TOPL.

[5]  Herbert Bos,et al.  Argos: an emulator for fingerprinting zero-day attacks for advertised honeypots with automatic signature generation , 2006, EuroSys.

[6]  David A. Wagner,et al.  A First Step Towards Automated Detection of Buffer Overrun Vulnerabilities , 2000, NDSS.

[7]  Jack W. Davidson,et al.  Secure and practical defense against code-injection attacks using software dynamic translation , 2006, VEE '06.

[8]  Andrew Warfield,et al.  Practical taint-based protection using demand emulation , 2006, EuroSys.

[9]  Miguel Castro,et al.  Preventing Memory Error Exploits with WIT , 2008, 2008 IEEE Symposium on Security and Privacy (sp 2008).

[10]  U. Bayer,et al.  TTAnalyze: A Tool for Analyzing Malware , 2006 .

[11]  John Johansen,et al.  PointGuard™: Protecting Pointers from Buffer Overflow Vulnerabilities , 2003, USENIX Security Symposium.

[12]  Nathanael Paul,et al.  Where's the FEEB? The Effectiveness of Instruction Set Randomization , 2005, USENIX Security Symposium.

[13]  Miguel Castro,et al.  Vigilante: end-to-end containment of internet worms , 2005, SOSP '05.

[14]  Eugene H. Spafford,et al.  The internet worm program: an analysis , 1989, CCRV.

[15]  Angelos D. Keromytis,et al.  On the General Applicability of Instruction-Set Randomization , 2010, IEEE Transactions on Dependable and Secure Computing.

[16]  David H. Ackley,et al.  Randomized instruction set emulation , 2005, TSEC.

[17]  Crispan Cowan,et al.  StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks , 1998, USENIX Security Symposium.

[18]  Angelos D. Keromytis,et al.  Countering code-injection attacks with instruction-set randomization , 2003, CCS '03.

[19]  James Newsome,et al.  Dynamic Taint Analysis for Automatic Detection, Analysis, and SignatureGeneration of Exploits on Commodity Software , 2005, NDSS.

[20]  Michael Owens.  Embedding an SQL database with SQLite , 2003 .

[21]  Dorothy E. Denning,et al.  A lattice model of secure information flow , 1976, CACM.

[22]  David Moore,et al.  Code-Red: a case study on the spread and victims of an internet worm , 2002, IMW '02.

[23]  Harish Patil,et al.  Pin: building customized program analysis tools with dynamic instrumentation , 2005, PLDI '05.

[24]  Daniel C. DuVarney,et al.  Efficient Techniques for Comprehensive Protection from Memory Error Exploits , 2005, USENIX Security Symposium.

[25]  A. One,et al.  Smashing The Stack For Fun And Profit , 1996 .

[26]  David A. Wagner,et al.  This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. Detecting Format String Vulnerabilities with Type Qualifiers , 2001 .

[27]  David H. Ackley,et al.  Randomized instruction set emulation to disrupt binary code injection attacks , 2003, CCS '03.

[28]  Donald F. Towsley,et al.  Code red worm propagation modeling and analysis , 2002, CCS '02.