New system security mechanisms for the cloud computing infrastructure

Cloud computing is a new computing paradigm, where computing resources are provided as a service to end users. To cut operating costs, hardware virtualization is used so that a single hardware platform is shared among multiple end users. This new paradigm introduces a complex security threat model, where threats can originate from external attackers, malicious end users or even the cloud infrastructure itself. In this dissertation, we introduce three novel security mechanisms that enhance the trustworthiness of cloud computing infrastructures: HIMA, HyperSentry, and SICE. HIMA, which stands for Hypervisor-based Integrity Measurement Agent, can be used to measure the integrity of guest VMs that run inside the cloud. This measurement can be further used to attest to the integrity of these guest VMs. Unlike other integrity measurement techniques, HIMA provides two essential capabilities: (1) Strong isolation between the measurement agent and the measurement target, and (2) Time of Check to Time of Use (TOCTTOU) consistency, which guarantees the continuity of the integrity evidence beyond the measurement time. HyperSentry is a novel framework to enable integrity measurement of running hypervisors (or any other highest privileged software). Unlike existing solutions for verifying privileged software, HyperSentry does not introduce a higher privileged software layer below the hypervisor. Instead, it introduces properly isolated software components that enable stealthy and in-context measurement of the runtime integrity of the hypervisor. While stealthiness is necessary to ensure that a compromised hypervisor does not have a chance to hide the attack traces upon detecting an up-coming measurement, in-context measurement is necessary to retrieve all the needed inputs for a successful integrity measurement. SICE, which stands for Strongly Isolated Computing Environment, is a novel framework to provide hardware-level isolation and protection for sensitive workloads running on x86 platforms in compute clouds. Unlike existing isolation techniques, SICE does not rely on any software component in the host environment (i.e., an OS or a hypervisor). Instead, the security of the isolated environments is guaranteed by a trusted computing base that only includes the hardware, the BIOS, and a very small System Management Mode (SMM) software foundation of about 300 lines of code.