Game theory based dynamic resource allocation for hybrid environment with cloud and big data application

Virtualization based cloud and big data applications have been widely adopted in various fields. Because deploying the big data applications on the cloud will cause obvious performance degradation, the cloud and big data applications are provided with fixed resource separately. However, the traditional fixed resource allocation mechanism has two drawbacks: (1) low resource utility and (2) unresponsiveness to the performance degradation. To address these drawbacks, the cloud and big data hybrid environment is designed, where fair resource allocation is used to ensure fairness between cloud and big data applications while virtual machine migration is used to make each virtual machine in cloud application reach its own satisfactory. Herein, game theory is used to model the conflict and negotiation between cloud and big data applications. Firstly, the Nash Equilibrium is used to discover the best strategy for both applications. Secondly, as for virtual machine migration, we use Nash Bargaining game to present the situation where virtual machines compete for more resources allocation while their minimal demand is ensured. Finally, experiments are carried out to prove that the hybrid environment outperforms the traditional method both in resource utility and application performance.

[1]  Hai Jin,et al.  Evaluating MapReduce on Virtual Machines: The Hadoop Case , 2009, CloudCom.

[2]  Haiyang Wang,et al.  On interference-aware provisioning for cloud-based big data processing , 2013, 2013 IEEE/ACM 21st International Symposium on Quality of Service (IWQoS).

[3]  Guangwen Yang,et al.  Load prediction using hybrid model for computational grid , 2007, 2007 8th IEEE/ACM International Conference on Grid Computing.

[4]  Phuong Nguyen,et al.  A Hybrid Scheduling Algorithm for Data Intensive Workloads in a MapReduce Environment , 2012, 2012 IEEE Fifth International Conference on Utility and Cloud Computing.

[5]  Yang Yang,et al.  Impacts of Virtualization Technologies on Hadoop , 2013, 2013 Third International Conference on Intelligent System Design and Engineering Applications.

[6]  Hai Jin,et al.  Falloc: Fair network bandwidth allocation in IaaS datacenters via a bargaining game approach , 2013, 2013 21st IEEE International Conference on Network Protocols (ICNP).

[7]  Fang Dong,et al.  Dynamic Resource Management in a HPC and Cloud Hybrid Environment , 2013, ICA3PP.

[8]  Xuedong Liang,et al.  Dynamic Spectrum Allocation in Wireless Cognitive Sensor Networks: Improving Fairness and Energy Efficiency , 2008, 2008 IEEE 68th Vehicular Technology Conference.

[9]  Shantenu Jha,et al.  Exploring application and infrastructure adaptation on hybrid grid-cloud infrastructure , 2010, HPDC '10.

[10]  Prasad Calyam,et al.  Defragmentation of Resources in Virtual Desktop Clouds for Cost-Aware Utility-Optimal Allocation , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[11]  Anthony A. Maciejewski,et al.  Static resource allocation for heterogeneous computing environments with tasks having dependencies, priorities, deadlines, and multiple versions , 2008, J. Parallel Distributed Comput..

[12]  Randy H. Katz,et al.  Above the Clouds: A Berkeley View of Cloud Computing , 2009 .