RandSys: Thwarting Code Injection Attacks with System Service Interface Randomization

Code injection attacks are a top threat to today's Internet. With zero-day attacks on the rise, randomization techniques have been introduced to diversify software and operation systems of networked hosts so that attacks that succeed on one host cannot succeed on others. Two most notable system-wide randomization techniques are instruction set randomization (ISR) and address space layout randomization (ASLR). The former randomizes instruction set for each process, while the latter randomizes the memory address space layout. Both suffer from a number of attacks. In this paper, we advocate and demonstrate that by combining ISR and ASLR effectively, we can offer much more robust protection than each of them individually. However, trivial combination of both schemes is not sufficient. To this end, we make the key observation that system call instructions matter the most to attackers for code injection. Our system, RandSys, uses system call instruction randomization and the general technique of ASLR along with a number of new enhancements to thwart code injection attacks. We have built a prototype for both Linux and Windows platforms. Our experiments show that RandSys can effectively thwart a wide variety of code injection attacks with a small overhead.

[1]  Christopher Krügel,et al.  Automating Mimicry Attacks Using Static Binary Analysis , 2005, USENIX Security Symposium.

[2]  David H. Ackley,et al.  Building diverse computer systems , 1997, Proceedings. The Sixth Workshop on Hot Topics in Operating Systems (Cat. No.97TB100133).

[3]  Niels Provos,et al.  Improving Host Security with System Call Policies , 2003, USENIX Security Symposium.

[4]  Hovav Shacham,et al.  On the effectiveness of address-space randomization , 2004, CCS '04.

[5]  Peter Szor,et al.  An Analysis of the Slapper Worm Ex-ploit , 2003 .

[6]  David A. Wagner,et al.  A First Step Towards Automated Detection of Buffer Overrun Vulnerabilities , 2000, NDSS.

[7]  Matt Pietrek,et al.  An in-depth look into the win32 portable executable le format , 2002 .

[8]  Helen J. Wang,et al.  Shield: vulnerability-driven network filters for preventing known vulnerability exploits , 2004, SIGCOMM.

[9]  Daniel C. DuVarney,et al.  Efficient Techniques for Comprehensive Protection from Memory Error Exploits , 2005, USENIX Security Symposium.

[10]  Dawn Song,et al.  Mitigating buffer overflows by operating system randomization , 2002 .

[11]  Navjot Singh,et al.  Transparent Run-Time Defense Against Stack-Smashing Attacks , 2000, USENIX Annual Technical Conference, General Track.

[12]  Crispin Cowan,et al.  FormatGuard: Automatic Protection From printf Format String Vulnerabilities , 2001, USENIX Security Symposium.

[13]  Nathanael Paul,et al.  Where's the FEEB? The Effectiveness of Instruction Set Randomization , 2005, USENIX Security Symposium.

[14]  Crispan Cowan,et al.  StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks , 1998, USENIX Security Symposium.

[15]  Angelos D. Keromytis,et al.  e-NeXSh: achieving an effectively non-executable stack and heap via system-call policing , 2005, 21st Annual Computer Security Applications Conference (ACSAC'05).

[16]  George C. Necula,et al.  CCured: type-safe retrofitting of legacy software , 2005, TOPL.

[17]  David H. Ackley,et al.  Randomized instruction set emulation to disrupt binary code injection attacks , 2003, CCS '03.

[18]  Derek Bruening,et al.  Secure Execution via Program Shepherding , 2002, USENIX Security Symposium.

[19]  David A. Wagner,et al.  MOPS: an infrastructure for examining security properties of software , 2002, CCS '02.

[20]  John Johansen,et al.  PointGuard™: Protecting Pointers from Buffer Overflow Vulnerabilities , 2003, USENIX Security Symposium.

[21]  Seung-Soon Im,et al.  Tool interface standard (TIS) executable and linking format (ELF) specification , 1995 .

[22]  Jun Xu,et al.  Non-Control-Data Attacks Are Realistic Threats , 2005, USENIX Security Symposium.

[23]  Christian S. Collberg,et al.  Protecting Against Unexpected System Calls , 2005, USENIX Security Symposium.

[24]  Daniel C. DuVarney,et al.  Address Obfuscation: An Efficient Approach to Combat a Broad Range of Memory Error Exploits , 2003, USENIX Security Symposium.

[25]  David E. Evans,et al.  Static detection of dynamic memory errors , 1996, PLDI '96.

[26]  Angelos D. Keromytis,et al.  Countering code-injection attacks with instruction-set randomization , 2003, CCS '03.

[27]  Iván Arce,et al.  An Analysis of the Slapper Worm , 2003, IEEE Secur. Priv..

[28]  Ravishankar K. Iyer,et al.  Transparent runtime randomization for security , 2003, 22nd International Symposium on Reliable Distributed Systems, 2003. Proceedings..

[29]  Angelos D. Keromytis,et al.  Building a Reactive Immune System for Software Services , 2005, USENIX Annual Technical Conference, General Track.

[30]  William A. Arbaugh,et al.  Copilot - a Coprocessor-based Kernel Runtime Integrity Monitor , 2004, USENIX Security Symposium.