Countering kernel malware in virtual execution environments

iii For my family iv ACKNOWLEDGEMENTS First of all, I wish to express my sincere thanks to my advisor, Dr. John Copeland for his insight guidance, endless patience and support. Without his help, this research could not have been completed. I am indebted to him. Dr. Alessandro (Alex) Orso and Dr Raheem Beyah for their valuable times, professional suggestions and gracious services on my committee. Myounghwan Lee for their valuable time and friendship. My most special thanks go to my parents Jiarang Xuan, Xiuying Zhu, and my wife Hu Tu for their everlasting encouragement, support and love.

[1]  J. Rutkowska.  Introducing Stealth Malware Taxonomy , 2006 .

[2]  William A. Arbaugh,et al.  Copilot - a Coprocessor-based Kernel Runtime Integrity Monitor , 2004, USENIX Security Symposium.

[3]  Raheem A. Beyah,et al.  Toward Revealing Kernel Malware Behavior in Virtual Execution Environments , 2009, RAID.

[4]  Eugene H. Spafford,et al.  The design and implementation of tripwire: a file system integrity checker , 1994, CCS '94.

[5]  Gerald J. Popek,et al.  Formal requirements for virtualizable third generation architectures , 1974, SOSP '73.

[6]  Raheem A. Beyah,et al.  Shepherding Loadable Kernel Modules through On-demand Emulation , 2009, DIMVA.

[7]  William A. Arbaugh,et al.  An Architecture for Specification-Based Detection of Semantic Integrity Violations in Kernel Dynamic Data , 2006, USENIX Security Symposium.

[8]  Jack W. Davidson,et al.  Safe virtual execution using software dynamic translation , 2002, 18th Annual Computer Security Applications Conference, 2002. Proceedings..

[9]  Zhenkai Liang,et al.  BitBlaze: A New Approach to Computer Security via Binary Analysis , 2008, ICISS.

[10]  Xuxian Jiang,et al.  Guest-Transparent Prevention of Kernel Rootkits with VMM-Based Memory Shadowing , 2008, RAID.

[11]  Nicholas Nethercote,et al.  Valgrind: A Program Supervision Framework , 2003, RV@CAV.

[12]  Zhenkai Liang,et al.  HookFinder: Identifying and Understanding Malware Hooking Behaviors , 2008, NDSS.

[13]  Arati Baliga,et al.  Lurking in the Shadows: Identifying Systemic Threats to Kernel Data , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[14]  J. Heasman.  Implementing and Detecting a PCI Rootkit , 2006 .

[15]  Yi-Min Wang,et al.  Detecting stealth software with Strider GhostBuster , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[16]  Andrea C. Arpaci-Dusseau,et al.  VMM-based hidden process detection and identification using Lycosid , 2008, VEE '08.

[17]  Samuel T. King,et al.  ReVirt: enabling intrusion analysis through virtual-machine logging and replay , 2002, OPSR.

[18]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[19]  Tal Garfinkel,et al.  A Virtual Machine Introspection Based Architecture for Intrusion Detection , 2003, NDSS.

[20]  Xiaoxin Chen,et al.  Overshadow: a virtualization-based approach to retrofitting protection in commodity operating systems , 2008, ASPLOS.

[21]  Xuxian Jiang,et al.  Mapping kernel objects to enable systematic integrity checking , 2009, CCS.

[22]  Tzi-cker Chiueh,et al.  A Forced Sampled Execution Approach to Kernel Rootkit Identification , 2007, RAID.

[23]  Adrian Perrig,et al.  SecVisor: a tiny hypervisor to provide lifetime kernel code integrity for commodity OSes , 2007, SOSP.

[24]  Fabrice Bellard,et al.  QEMU, a Fast and Portable Dynamic Translator , 2005, USENIX ATC, FREENIX Track.

[25]  Tal Garfinkel,et al.  Terra: a virtual machine-based platform for trusted computing , 2003, SOSP '03.

[26]  Elaine Shi,et al.  Pioneer: verifying code integrity and enforcing untampered code execution on legacy systems , 2005, SOSP '05.

[27]  Ken Chiang,et al.  A Case Study of the Rustock Rootkit and Spam Bot , 2007, HotBots.

[28]  Beng-Hong Lim,et al.  Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor , 2001, USENIX Annual Technical Conference, General Track.

[29]  Andrew Warfield,et al.  Practical taint-based protection using demand emulation , 2006, EuroSys.

[30]  Felix C. Freiling,et al.  Return-Oriented Rootkits: Bypassing Kernel Code Integrity Protection Mechanisms , 2009, USENIX Security Symposium.

[31]  Cynthia E. Irvine,et al.  Analysis of the Intel Pentium's Ability to Support a Secure Virtual Machine Monitor , 2000, USENIX Security Symposium.

[32]  Michael W. Hicks,et al.  Automated detection of persistent kernel control-flow attacks , 2007, CCS '07.

[33]  Gil Neiger,et al.  Intel virtualization technology , 2005, Computer.

[34]  Wenke Lee,et al.  K-Tracer: A System for Extracting Kernel Malware Behavior , 2009, NDSS.

[35]  Min Gyung Kang,et al.  Emulating emulation-resistant malware , 2009, VMSec '09.

[36]  Peter Ferrie.  Attacks on Virtual Machine Emulators , 2007 .

[37]  Christopher Krügel,et al.  Detecting kernel-level rootkits through binary analysis , 2004, 20th Annual Computer Security Applications Conference.

[38]  Christopher Krügel,et al.  Exploring Multiple Execution Paths for Malware Analysis , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[39]  Arati Baliga,et al.  Automatic Inference and Enforcement of Kernel Data Structure Invariants , 2008, 2008 Annual Computer Security Applications Conference (ACSAC).

[40]  Xuxian Jiang,et al.  Multi-aspect profiling of kernel rootkit behavior , 2009, EuroSys '09.

[41]  Xuxian Jiang,et al.  Stealthy malware detection through vmm-based "out-of-the-box" semantic view reconstruction , 2007, CCS '07.

[42]  Performance Evaluation of Intel EPT Hardware Assist , 2006 .

[43]  Hovav Shacham,et al.  The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86) , 2007, CCS '07.

[44]  Mihai Budiu,et al.  Control-flow integrity principles, implementations, and applications , 2009, TSEC.

[45]  Roy H. Campbell,et al.  Cloaker: Hardware Supported Rootkit Concealment , 2008, 2008 IEEE Symposium on Security and Privacy (sp 2008).

[46]  Xuxian Jiang,et al.  Countering kernel rootkits with lightweight hook protection , 2009, CCS.

[47]  Abhinav Srivastava,et al.  Robust signatures for kernel data structures , 2009, CCS.

[48]  David Lie,et al.  Hypervisor Support for Identifying Covertly Executing Binaries , 2008, USENIX Security Symposium.

[49]  Helen J. Wang,et al.  SubVirt: implementing malware with virtual machines , 2006, 2006 IEEE Symposium on Security and Privacy (S&P'06).

[50]  Derek Bruening,et al.  Secure Execution via Program Shepherding , 2002, USENIX Security Symposium.

[51]  Tal Garfinkel,et al.  VMwareDecoupling Dynamic Program Analysis from Execution in Virtual Environments , 2008, USENIX Annual Technical Conference.