Supercloud

Infrastructure-as-a-Service (IaaS) cloud providers hide available interfaces for virtual machine (VM) placement and migration, CPU capping, memory ballooning, page sharing, and I/O throttling, limiting the ways in which applications can optimally configure resources or respond to dynamically shifting workloads. Given these interfaces, applications could migrate VMs in response to diurnal workloads or changing prices, adjust resources in response to load changes, and so on. This article proposes a new abstraction that we call a Library Cloud and that allows users to customize the diverse available cloud resources to best serve their applications. We built a prototype of a Library Cloud that we call the Supercloud. The Supercloud encapsulates applications in a virtual cloud under users’ full control and can incorporate one or more availability zones within a cloud provider or across different providers. The Supercloud provides virtual machine, storage, and networking complete with a full set of management operations, allowing applications to optimize performance. In this article, we demonstrate various innovations enabled by the Library Cloud.

[1]  Jon Crowcroft,et al.  Unikernels: library operating systems for the cloud , 2013, ASPLOS '13.

[2]  E. Michael Maximilien,et al.  IBM altocumulus: a cross-cloud middleware and platform , 2009, OOPSLA Companion.

[3]  Anant Agarwal,et al.  An operating system for multicore and clouds: mechanisms and implementation , 2010, SoCC '10.

[4]  Dawson R. Engler,et al.  Exokernel: an operating system architecture for application-level resource management , 1995, SOSP.

[5]  Anja Feldmann,et al.  Live wide-area migration of virtual machines including local persistent state , 2007, VEE '07.

[6]  Donald E. Porter,et al.  Rethinking the library OS from the top down , 2011, ASPLOS XVI.

[7]  Jeffrey S. Chase,et al.  Automated control for elastic storage , 2010, ICAC '10.

[8]  Xiaohui Gu,et al.  CloudScale: elastic resource scaling for multi-tenant cloud systems , 2011, SoCC.

[9]  Wei Jin,et al.  USENIX Association Proceedings of USITS ’ 03 : 4 th USENIX Symposium on Internet Technologies and Systems , 2003 .

[10]  Matei Ripeanu,et al.  VMFlock: virtual machine co-migration for the cloud , 2011, HPDC '11.

[11]  Carl A. Waldspurger,et al.  Memory resource management in VMware ESX server , 2002, OSDI '02.

[12]  Jan Broeckhove,et al.  Nesting Virtual Machines in Virtualization Test Frameworks , 2010 .

[13]  Damian L. Osisek,et al.  ESA/390 Interpretive-Execution Architecture, Foundation for VM/ESA , 1991, IBM Syst. J..

[14]  Robbert van Renesse,et al.  Supercloud: Opportunities and Challenges , 2015, OPSR.

[15]  Christopher Stewart,et al.  A Dollar from 15 Cents: Cross-Platform Management for Internet Services , 2008, USENIX Annual Technical Conference.

[16]  Prateek Sharma,et al.  SpotCheck: designing a derivative IaaS cloud on the spot market , 2015, EuroSys.

[17]  Haipeng Luo,et al.  Adaptive Resource Provisioning for the Cloud Using Online Bin Packing , 2014, IEEE Transactions on Computers.

[18]  Tal Garfinkel,et al.  The Design and Evolution of Live Storage Migration in VMware ESX , 2011, USENIX Annual Technical Conference.

[19]  Chunqiang Tang,et al.  FVD: A High-Performance Virtual Machine Image Format for Cloud , 2011, USENIX Annual Technical Conference.

[20]  Muli Ben-Yehuda,et al.  The Turtles Project: Design and Implementation of Nested Virtualization , 2010, OSDI.

[21]  Pramod Bhatotia,et al.  Orchestrating the Deployment of Computations in the Cloud with Conductor , 2012, NSDI.

[22]  Shrisha Rao,et al.  CloudSpider: Combining Replication with Scheduling for Optimizing Live Migration of Virtual Machines across Wide Area Networks , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[23]  Dan Dobre,et al.  Hybris: Robust Hybrid Cloud Storage , 2014, SoCC.

[24]  Jie Zheng,et al.  Workload-aware live storage migration for clouds , 2011, VEE '11.

[25]  Miguel Correia,et al.  DepSky: Dependable and Secure Storage in a Cloud-of-Clouds , 2013, TOS.

[26]  Heeseung Jo,et al.  Task-aware virtual machine scheduling for I/O performance. , 2009, VEE '09.

[27]  Zhen Xiao,et al.  Dynamic Resource Allocation Using Virtual Machines for Cloud Computing Environment , 2013, IEEE Transactions on Parallel and Distributed Systems.

[28]  James E. Smith,et al.  The architecture of virtual machines , 2005, Computer.

[29]  Lucas Chaufournier,et al.  CloudNet: Dynamic Pooling of Cloud Resources by Live WAN Migration of Virtual Machines , 2011, IEEE/ACM Transactions on Networking.

[30]  George Varghese,et al.  Difference engine , 2010, OSDI.

[31]  Ethan Katz-Bassett,et al.  SPANStore: cost-effective geo-replicated storage spanning multiple cloud services , 2013, SOSP.

[32]  Steven Hand,et al.  Self-adaptive and self-configured CPU resource provisioning for virtualized servers using Kalman filters , 2009, ICAC '09.

[33]  Robert P. Goldberg,et al.  Survey of virtual machine research , 1974, Computer.

[34]  Mike Hibler,et al.  Microkernels meet recursive virtual machines , 1996, OSDI '96.

[35]  Gerald Belpaire,et al.  Formal properties of recursive Virtual Machine architectures. , 1975, SOSP.

[36]  Hakim Weatherspoon,et al.  The Xen-Blanket: virtualize once, run everywhere , 2012, EuroSys '12.

[37]  Suman Nath,et al.  Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services , 2008, NSDI.

[38]  Gerald Belpaire,et al.  Hardware architecture for recursive Virtual Machines , 1975, ACM '75.

[39]  Prashant J. Shenoy,et al.  Dynamic resource allocation for shared data centers using online measurements , 2003, IWQoS'03.

[40]  Gil Neiger,et al.  Intel virtualization technology , 2005, Computer.

[41]  David Walker,et al.  Frenetic: a high-level language for OpenFlow networks , 2010, PRESTO '10.

[42]  Butler W. Lampson,et al.  Hints for Computer System Design , 1983, IEEE Software.

[43]  Kang G. Shin,et al.  Automated control of multiple virtualized resources , 2009, EuroSys '09.

[44]  Satoshi Sekiguchi,et al.  A live storage migration mechanism over wan and its performance evaluation , 2009, VTDC '09.

[45]  Jerome A. Rolia,et al.  Workload Analysis and Demand Prediction of Enterprise Data Center Applications , 2007, 2007 IEEE 10th International Symposium on Workload Characterization.

[46]  Xiaoyun Zhu,et al.  1000 Islands: Integrated Capacity and Workload Management for the Next Generation Data Center , 2008, 2008 International Conference on Autonomic Computing.

[47]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[48]  Eyal de Lara,et al.  SnowFlock: rapid virtual machine cloning for cloud computing , 2009, EuroSys '09.

[49]  Kartik Gopalan,et al.  Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning , 2009, VEE '09.

[50]  Prashant J. Shenoy,et al.  Seagull: Intelligent Cloud Bursting for Enterprise Applications , 2012, USENIX Annual Technical Conference.

[51]  Peter Desnoyers,et al.  HIL: Designing an Exokernel for the Data Center , 2016, SoCC.

[52]  Franck Cappello,et al.  A hybrid local storage transfer scheme for live migration of I/O intensive workloads , 2012, HPDC '12.

[53]  Alexander Shraer,et al.  Dynamic Reconfiguration of Primary/Backup Clusters , 2012, USENIX Annual Technical Conference.

[54]  Flavio Paiva Junqueira,et al.  Zab: High-performance broadcast for primary-backup systems , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[55]  Ramakrishna Kotla,et al.  SafeStore: A Durable and Practical Storage System , 2007, USENIX Annual Technical Conference.

[56]  A. Mirkin.  Containers checkpointing and live migration , 2010 .

[57]  Hugh C. Lauer,et al.  A recursive virtual machine architecture , 1973 .

[58]  Arun Venkataramani,et al.  Black-box and Gray-box Strategies for Virtual Machine Migration , 2007, NSDI.

[59]  Westone,et al.  Home Page , 2004, 2022 2nd International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA).

[60]  Miguel Correia,et al.  SCFS: A Shared Cloud-backed File System , 2014, USENIX Annual Technical Conference.

[61]  Tal Garfinkel,et al.  Compatibility Is Not Transparency: VMM Detection Myths and Realities , 2007, HotOS.

[62]  Gerald J. Popek,et al.  Formal requirements for virtualizable third generation architectures , 1974, SOSP '73.

[63]  Ari Juels,et al.  HAIL: a high-availability and integrity layer for cloud storage , 2009, CCS.

[64]  Xiaohui Gu,et al.  AGILE: Elastic Distributed Resource Scaling for Infrastructure-as-a-Service , 2013, ICAC.

[65]  Alysson Neves Bessani,et al.  The TClouds architecture: Open and resilient cloud-of-clouds computing , 2012, IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN 2012).

[66]  Haibo Chen,et al.  CloudVisor: retrofitting protection of virtual machines in multi-tenant cloud with nested virtualization , 2011, SOSP.

[67]  Jerome H. Saltzer,et al.  End-to-end arguments in system design , 1984, TOCS.

[68]  Xavier Lorca,et al.  Entropy: a consolidation manager for clusters , 2009, VEE '09.

[69]  Eyal de Lara,et al.  Kaleidoscope: cloud micro-elasticity via VM state coloring , 2011, EuroSys '11.

[70]  Larry L. Peterson,et al.  Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors , 2007, EuroSys '07.

[71]  Hakim Weatherspoon,et al.  RACS: a case for cloud storage diversity , 2010, SoCC '10.