Towards Superclouds

Cloud computing has emerged as an economically attractive utility model for computational resources. An increasing number of industries, from businesses to governments, are embracing cloud computing. However, the economic benefits of the cloud computing model come at a price: the loss of control over how services and applications can use computing resources. In other words, cloud computing is fundamentally provider-centric. Cloud providers (the providers of computational resources), not cloud users (the consumers of computational resources), dictate rules and policies governing how computational resources can be used. For large enterprise application workloads, adhering to cloud provider rules and policies may be prohibitive. This dissertation explores the question: how can large enterprise workloads efficiently utilize and control computational resources from a variety of providers in the cloud computing model? The main contributions of this dissertation relate to a fundamental change to the cloud computing model. Instead of a provider-centric model, we propose a user-centric model in which the cloud user can maintain control over how computational resources obtained from cloud providers can be used. We have devised a new abstraction, called cloud extensibility , to enable the implementation of provider-level functionality by cloud users. Leveraging cloud extensibility, we describe steps towards a user-centric cloud computing model that grants cloud users—including large enterprises—control over resources obtained from one or more cloud providers. We call this new model the supercloud model. More specifically, we focus on three key areas in which current provider-centric cloud computing models do not expose the necessary control or lack the features to support large enterprise workloads without significant reconfiguration effort. First, clouds are not interoperable, restricting workloads to a single provider and hindering incremental migration to the cloud. Second, clouds lack support for complex enterprise network configurations, including flow policies between application components and low-level network features (e.g., IP addresses, multicast, VLANs). Finally, high utilization of cloud resources cannot be applied through techniques like oversubscription, and existing techniques do not apply well to common workload patterns. We subsequently make three contributions, embodied in the design, implementation and evaluation of three systems that leverage cloud extensibility. Cloud extensibility itself is instantiated in the first system, a nested virtualization layer called the Xen-Blanket. The Xen-Blanket additionally enables cloud interoperability by homogenizing existing cloud interfaces and services. The second system, VirtualWire, provides a virtual network abstraction to support complex enterprise networks in which the cloud user manages the network control logic. Finally, we present Overdriver , a system that enables high resource utilization through memory oversubscription and the handling of the resulting—often transient and unpredictable—memory overload. Together, these three systems are important steps towards superclouds.