HDID: An Efficient Hybrid Docker Image Distribution System for Datacenters

Docker, a container-based virtualization technology, is widely used in datacenters. To increase the speed of booting containers, the storage and the distribution of Docker images have received much research effort. There are also some efforts which use Peer-to-Peer file downloading method to speed-up Docker image distribution. However, these systems just package image as a file for sharing without considering the layering structure of the Docker image. To increase the booting speed of Docker image, in this work, we propose an efficient Hybrid Docker Image Distribution System for Datacenters (HDID). First, to gain insight into the layer structure of Docker images, we investigate the size distribution of the top 29 popular Docker image layers. Second, based on the study of the size distribution, we introduce a hybrid distribution method which adaptively uses both the BitTorrent protocol, a P2P file downloading technique, and the original Registry downloading technique to boost the image transferring procedure. Third, as the BitTorrent protocol is designed for the Internet instead of datacenter networks, to further reduce the booting time, we develop file-downloading strategy, which optimizes BitTorrent for Docker images in datacenter networks. Finally, through extensive experiments, we show that the proposed method can reduce the image distribution time of Docker by about 90% compared to the default Docker image distribution method.