CoMICon: A Co-Operative Management System for Docker Container Images

Docker containers are becoming an attractive implementation choice for next-generation microservices-based applications. When provisioning such an application, container (microservice) instances need to be created from individual container images. Starting a container on a node, where images are locally available, is fast but it may not guarantee the quality of service due to insufficient resources. When a collection of nodes are available, one can select a node with sufficient resources. However, if the selected node does not have the required image, downloading the image from a different registry increases the provisioning time. Motivated by these observations, in this paper, we present CoMICon, a system for co-operative management of Docker images among a set of nodes. The key features of CoMICon are: (1) it enables a co-operative registry among a set of nodes, (2) it can store or delete images partially in the form of layers, (3) it facilitates the transfer of image layers between registries, and (4) it enables distributed pull of an image while starting a container. Using these features, we describe—(i) high availability management of images and (ii) provisioning management of distributed microservices based applications. We extensively evaluate the performance of CoMICon using 142 real, publicly available images from Docker hub. In contrast to state-of-the-art full image based approach, CoMICon can increase the number of highly available images up to 3x while reducing the application provisioning time by 28% on average.

[1]  Akshat Verma,et al.  12MAP: Cloud Disaster Recovery Based on Image-Instance Mapping , 2013, Middleware.

[2]  Thilo Kielmann,et al.  Squirrel: scatter hoarding VM image contents on IaaS compute nodes , 2014, HPDC '14.

[3]  Thilo Kielmann,et al.  Prebaked µVMs: Scalable, Instant VM Startup for IaaS Clouds , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[4]  Emin Gün Sirer,et al.  Tiered Replication: A Cost-effective Alternative to Full Cluster Geo-replication , 2015, USENIX Annual Technical Conference.

[5]  Kui Wu,et al.  VMThunder: Fast Provisioning of Large-Scale Virtual Machine Clusters , 2014, IEEE Transactions on Parallel and Distributed Systems.

[6]  Gang Huang,et al.  Model-based high availability configuration framework for cloud , 2013, MDS '13.

[7]  Sebastien Goasguen,et al.  Image Distribution Mechanisms in Large Scale Cloud Providers , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[8]  Vishal Misra,et al.  VMTorrent: scalable P2P virtual machine streaming , 2012, CoNEXT '12.

[9]  Bogdan Nicolae,et al.  Discovering and Leveraging Content Similarity to Optimize Collective on-Demand Data Access to IaaS Cloud Storage , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[10]  Chris M. O'Donnell Using BitTorrent to distribute virtual machine images for classes , 2008, SIGUCCS '08.

[11]  Larry L. Peterson,et al.  Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors , 2007, EuroSys '07.

[12]  Mazin S. Yousif,et al.  Microservices , 2016, IEEE Cloud Comput..

[13]  Carlo Curino,et al.  OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases , 2013, Proc. VLDB Endow..

[14]  Darrell Reimer,et al.  Virtual Machine Images as Structured Data: The Mirage Image Library , 2011, HotCloud.

[15]  Kang G. Shin,et al.  Performance Evaluation of Virtualization Technologies for Server Consolidation , 2007 .

[16]  Sachin Katti,et al.  Copysets: Reducing the Frequency of Data Loss in Cloud Storage , 2013, USENIX Annual Technical Conference.

[17]  Andrea C. Arpaci-Dusseau,et al.  Slacker: Fast Distribution with Lazy Docker Containers , 2016, FAST.

[18]  Cory Hill,et al.  f4: Facebook's Warm BLOB Storage System , 2014, OSDI.

[19]  Tridib Mukherjee,et al.  SCoPe: A Decision System for Large Scale Container Provisioning Management , 2016, 2016 IEEE 9th International Conference on Cloud Computing (CLOUD).

[20]  Gabriel Antoniu,et al.  Going back and forth: efficient multideployment and multisnapshotting on clouds , 2011, HPDC '11.