On the Resource Management of Kubernetes

Kubernetes is the most popular container orchestration platform that enables users to create and run multiple containers in cloud environments. Kubernetes offers resource management to isolate the resource usage of containers on a host server because performance isolation is an important factor in terms of service quality. This paper investigates whether the resource management of Kubernetes is sufficient to isolate the performance of containers. This is different from previous studies that mostly focuses on efficient resource management rather than the study on performance interference. We evaluate the performance interference 1) between CPU-intensive and networkintensive containers, and 2) between multiple network-intensive containers. Our evaluation results show that containers experience performance degradation by 50% due to the co-located containers even under the resource management of Kubernetes. This paper also points out that the root cause of the performance interference between multiple network-intensive containers is CPU contention, not network bandwidth. As a result, Kubernetes needs to consider the CPU usage of network-related workloads in resource management in order to mitigate the performance interference.

[1]  Wouter Joosen,et al.  A Comprehensive Feature Comparison Study of Open-Source Container Orchestration Frameworks , 2019, Applied Sciences.

[2]  Bo Li,et al.  Maximizing container-based network isolation in parallel computing clusters , 2016, 2016 IEEE 24th International Conference on Network Protocols (ICNP).

[3]  Kevin Skadron,et al.  Bubble-up: Increasing utilization in modern warehouse scale computers via sensible co-locations , 2011, 2011 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[4]  Albert G. Greenberg,et al.  EyeQ: Practical Network Performance Isolation at the Edge , 2013, NSDI.

[5]  Abhishek Verma,et al.  Large-scale cluster management at Google with Borg , 2015, EuroSys.

[6]  Rajkumar Buyya,et al.  Container‐based cluster orchestration systems: A taxonomy and future directions , 2018, Softw. Pract. Exp..

[7]  Cong Xu,et al.  Iron: Isolating Network-based CPU in Container Environments , 2018, NSDI.

[8]  Cheol-Ho Hong,et al.  qCon: QoS-Aware Network Resource Management for Fog Computing , 2018, Sensors.

[9]  Omer F. Rana,et al.  Adaptive Application Scheduling under Interference in Kubernetes , 2016, 2016 IEEE/ACM 9th International Conference on Utility and Cloud Computing (UCC).

[10]  Cong Xu,et al.  NBWGuard: Realizing Network QoS for Kubernetes , 2018, Middleware Industry.

[11]  Xiao Zhang,et al.  CPI2: CPU performance isolation for shared compute clusters , 2013, EuroSys '13.

[12]  Eric A. Brewer,et al.  Borg, Omega, and Kubernetes , 2016, ACM Queue.

[13]  Tan N. Le,et al.  AlloX: compute allocation in hybrid clusters , 2020, EuroSys.

[14]  Amin Vahdat,et al.  SENIC: Scalable NIC for End-Host Rate Limiting , 2014, NSDI.

[15]  Randy H. Katz,et al.  Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center , 2011, NSDI.

[16]  James Cheng,et al.  Improving resource utilization by timely fine-grained scheduling , 2020, EuroSys.