Prediction-based virtual instance migration for balanced workload in the cloud datacenters

Datacenters in the cloud today provide virtualized resources of CPU, memory, disk, and networks so that millions of users can use the services at the same time in an efficient and scalable way. One of the major challenges in these datacenters is load balancing and shifting. As a huge number of requests are sent to a particular datacenter or a group of servers are asked to process more than their fair share, some of the servers are overloaded, slowed down, hot spots are created, and even hardware failures may occur. This unbalanced load in the end deteriorates the performance of the entire system easily. In this paper, we propose a load balancer that aims at alleviating hot spots and distributing the load from overloaded servers to underutilized servers. Our load balancer monitors the loads of the servers, detects indications of overloading, then migrates virtual instances from overloaded servers to target servers. We have implemented the load balancer in a real system using the Xen hypervisor. We have also conducted an event-driven simulation to evaluate the performance of our system on a large-scale. Our results indicate that our reactive-predictive load balancing algorithm helps balance load among servers in the cloud as much as the best-case scenario from the exhaustive search with much less overhead.