Implementation and evaluation of a container management platform on Docker: Hadoop deployment as an example

In recent years, virtualization is one of the key technologies of next-generation data centers. However, the problem of virtualization technology is that each instance needs to run a client operating system and a lot of applications. Therefore, it might generate a heavy load and affect the system efficiency and performance. In this work, the performance evaluation of three environments (bare-metal, Docker containers, and virtual machines) is investigated to understand the differences between the characteristics of each environment. Also, we addressed whether container-based virtualization can solve the problems of traditional virtualization. In addition, we combined Docker with OpenStack to implement a container management platform. Finally, we took Hadoop deployment as an example to verify whether Docker can solve the deployment problem and save time.

[1]  Tomáš Dulík,et al.  Docker as Platform for Assignments Evaluation , 2015 .

[2]  Yoji Yamato,et al.  Development of resource management server for production IaaS services based on OpenStack , 2015, J. Inf. Process..

[3]  Chao-Tung Yang,et al.  Virtual machine management system based on the power saving algorithm in cloud , 2017, J. Netw. Comput. Appl..

[4]  Chao-Tung Yang,et al.  Implementation of an energy saving cloud infrastructure with virtual machine power usage monitoring and live migration on OpenStack , 2020, Computing.

[5]  Y. C. Tay,et al.  A Performance Comparison of Containers and Virtual Machines in Workload Migration Context , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW).

[6]  Larry L. Peterson,et al.  Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors , 2007, EuroSys '07.

[7]  Ling Liu,et al.  A Comparative Study of Containers and Virtual Machines in Big Data Environment , 2018, 2018 IEEE 11th International Conference on Cloud Computing (CLOUD).

[8]  Chao-Tung Yang,et al.  An energy-efficient cloud system with novel dynamic resource allocation methods , 2019, The Journal of Supercomputing.

[9]  Lars Lundberg,et al.  Performance evaluation of container and virtual machine running cassandra workload , 2017, 2017 3rd International Conference of Cloud Computing Technologies and Applications (CloudTech).

[10]  P. Dhavachelvan,et al.  Big Data and Hadoop-a Study in Security Perspective , 2015 .

[11]  Antonio Brogi,et al.  Cloud Container Technologies: A State-of-the-Art Review , 2019, IEEE Transactions on Cloud Computing.

[12]  Shuichi Oikawa,et al.  Behavior-Based Memory Resource Management for Container-Based Virtualization , 2016, 2016 4th Intl Conf on Applied Computing and Information Technology/3rd Intl Conf on Computational Science/Intelligence and Applied Informatics/1st Intl Conf on Big Data, Cloud Computing, Data Science & Engineering (ACIT-CSII-BCD).

[13]  Yousof Al-Hammadi,et al.  Performance comparison between container-based and VM-based services , 2017, 2017 20th Conference on Innovations in Clouds, Internet and Networks (ICIN).

[14]  Patrick Martineau,et al.  Benchmarking Hadoop Performance in the Cloud - An in Depth Study of Resource Management and Energy Consumption , 2016, CLOSER.

[15]  Junzo Watada,et al.  Emerging Trends, Techniques and Open Issues of Containerization: A Review , 2019, IEEE Access.

[16]  Di Liu,et al.  The research and implementation of cloud computing platform based on docker , 2014, 2014 11th International Computer Conference on Wavelet Actiev Media Technology and Information Processing(ICCWAMTIP).

[17]  Gábor Terstyánszky,et al.  Buttressing volatile desktop grids with cloud resources within a reconfigurable environment service for workflow orchestration , 2014, Journal of Cloud Computing.

[18]  Ramakrishnan Rajamony,et al.  An updated performance comparison of virtual machines and Linux containers , 2015, 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[19]  Keqiu Li,et al.  A generic cloud platform for engineering optimization based on OpenStack , 2014, Adv. Eng. Softw..

[20]  Martín Casado,et al.  Extending Networking into the Virtualization Layer , 2009, HotNets.

[21]  Algirdas Maknickas,et al.  Development of cloud services for patient-specific simulations of blood flows through aortic valves , 2017, Adv. Eng. Softw..

[22]  Ashish Lingayat,et al.  Performance Evaluation for Deploying Docker Containers On Baremetal and Virtual Machine , 2018, 2018 3rd International Conference on Communication and Electronics Systems (ICCES).

[23]  Hai Jin,et al.  A novel memory allocation scheme for memory energy reduction in virtualization environment , 2015, J. Comput. Syst. Sci..

[24]  Mohammed Moin Mulla,et al.  Performance Evaluation of Docker Container and Virtual Machine , 2020 .

[25]  E. T. G. Sousa,et al.  Performance Comparison Between Virtual Machines and Docker Containers , 2018, IEEE Latin America Transactions.

[26]  Rajkumar Buyya,et al.  Mastering Cloud Computing: Foundations and Applications Programming , 2013 .

[27]  Patrick Martineau,et al.  Experimental Study on Performance and Energy Consumption of Hadoop in Cloud Environments , 2016, CLOSER.

[28]  Haibing Guan,et al.  HYVI: A HYbrid VIrtualization Solution Balancing Performance and Manageability , 2014, IEEE Transactions on Parallel and Distributed Systems.

[29]  Rabindra K. Barik,et al.  Performance analysis of virtual machines and containers in cloud computing , 2016, 2016 International Conference on Computing, Communication and Automation (ICCCA).

[30]  Yoji Yamato,et al.  Development of template management technology for easy deployment of virtual resources on OpenStack , 2014, Journal of Cloud Computing.