Security implications of memory deduplication in a virtualized environment

Memory deduplication has been widely used in various commodity hypervisors. By merging identical memory contents, it allows more virtual machines to run concurrently on top of a hypervisor. However, while this technique improves memory efficiency, it has a large impact on system security. In particular, memory deduplication is usually implemented using a variant of copy-on-write techniques, for which, writing to a shared page would incur a longer access time than those non-shared. In this paper, we investigate the security implication of memory deduplication from the perspectives of both attackers and defenders. On one hand, using the artifact above, we demonstrate two new attacks to create a covert channel and detect virtualization, respectively. On the other hand, we also show that memory deduplication can be leveraged to safeguard Linux kernel integrity.

[1]  Zhi Wang,et al.  Countering Persistent Kernel Rootkits through Systematic Hook Discovery , 2008, RAID.

[2]  Michal Kaczmarczyk,et al.  HYDRAstor: A Scalable Secondary Storage , 2009, FAST.

[3]  Cyrille Artho,et al.  Moving from Logical Sharing of Guest OS to Physical Sharing of Deduplication on Virtual Machine , 2010, HotSec.

[4]  William A. Arbaugh,et al.  Copilot - a Coprocessor-based Kernel Runtime Integrity Monitor , 2004, USENIX Security Symposium.

[5]  Steven Hand,et al.  Satori: Enlightened Page Sharing , 2009, USENIX Annual Technical Conference.

[6]  Markus Jakobsson,et al.  Crimeware: Understanding New Attacks and Defenses , 2008 .

[7]  Manoj B. Athreya.  Subverting Linux on-the-fly using hardware virtualization technology , 2010 .

[8]  Arati Baliga,et al.  Detecting Kernel-Level Rootkits Using Data Structure Invariants , 2011, IEEE Transactions on Dependable and Secure Computing.

[9]  Cyrille Artho,et al.  Memory deduplication as a threat to the guest OS , 2011, EUROSEC '11.

[10]  Christopher Thompson,et al.  Virtualization Detection : New Strategies and Their Effectiveness , 2010 .

[11]  Peng Ning,et al.  HIMA: A Hypervisor-Based Integrity Measurement Agent , 2009, 2009 Annual Computer Security Applications Conference.

[12]  Butler W. Lampson,et al.  A note on the confinement problem , 1973, CACM.

[13]  Zhenyu Wu,et al.  Whispers in the Hyper-space: High-speed Covert Channel Attacks in the Cloud , 2012, USENIX Security Symposium.

[14]  Jonathon T. Giffin,et al.  2011 IEEE Symposium on Security and Privacy Virtuoso: Narrowing the Semantic Gap in Virtual Machine Introspection , 2022 .

[15]  Benny Pinkas,et al.  Side Channels in Cloud Services: Deduplication in Cloud Storage , 2010, IEEE Security & Privacy.

[16]  Tal Garfinkel,et al.  A Virtual Machine Introspection Based Architecture for Intrusion Detection , 2003, NDSS.

[17]  Edgar R. Weippl,et al.  Dark Clouds on the Horizon: Using Cloud Storage as Attack Vector and Online Slack Space , 2011, USENIX Security Symposium.

[18]  Carl A. Waldspurger,et al.  Memory resource management in VMware ESX server , 2002, OSDI '02.

[19]  Hovav Shacham,et al.  Hey, you, get off of my cloud: exploring information leakage in third-party compute clouds , 2009, CCS.

[20]  Adrian Perrig,et al.  Towards Sound Detection of Virtual Machines , 2008, Botnet Detection.

[21]  Xuxian Jiang,et al.  Guest-Transparent Prevention of Kernel Rootkits with VMM-Based Memory Shadowing , 2008, RAID.

[22]  Xuxian Jiang,et al.  Stealthy malware detection through vmm-based "out-of-the-box" semantic view reconstruction , 2007, CCS '07.

[23]  Cyrille Artho,et al.  Software Side Channel Attack on Memory Deduplication , 2011, SOSP 2011.

[24]  Matti A. Hiltunen,et al.  An exploration of L2 cache covert channels in virtualized environments , 2011, CCSW '11.

[25]  Dongyan Xu,et al.  Polymorphing Software by Randomizing Data Structure Layout , 2009, DIMVA.

[26]  Darrell D. E. Long,et al.  Deep Store: an archival storage system architecture , 2005, 21st International Conference on Data Engineering (ICDE'05).

[27]  Tal Garfinkel,et al.  Compatibility Is Not Transparency: VMM Detection Myths and Realities , 2007, HotOS.