Resource intensity aware job scheduling in a distributed cloud

A Dominant Resource Fairness (DRF) based scheme for job scheduling in distributed cloud computing systems which was modeled as multi-job scheduling and multi-resource allocation coupling problem is proposed, where the resource pool is constructed from a large number of distributed heterogeneous servers, representing different points in the configuration space of resources such as processing, memory, storage and bandwidth. By introducing dominant resource share of jobs and virtual machines, the multi-job scheduling and multi-resource allocation joint mechanism significantly improves the cloud system's resource utilization, yet with a substantial reduction of job completion times. We show through experiments and case studies the superior performance of the algorithms in practice.

[1]  R. Srikant,et al.  Stochastic models of load balancing and scheduling in cloud computing clusters , 2012, 2012 Proceedings IEEE INFOCOM.

[2]  Moustafa Ghanem,et al.  Elastic Application Container: A Lightweight Approach for Cloud Resource Provisioning , 2012, 2012 IEEE 26th International Conference on Advanced Information Networking and Applications.

[3]  Benjamin Hindman,et al.  Dominant Resource Fairness: Fair Allocation of Multiple Resource Types , 2011, NSDI.

[4]  Andrew V. Goldberg,et al.  Quincy: fair scheduling for distributed computing clusters , 2009, SOSP '09.

[5]  Scott Shenker,et al.  Choosy: max-min fair sharing for datacenter jobs with constraints , 2013, EuroSys '13.

[6]  Hadas Shachnai,et al.  Scheduling jobs with dwindling resource requirements in clouds , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[7]  Nathan Linial,et al.  No justified complaints: on fair sharing of multiple resources , 2011, ITCS '12.

[8]  Noam Nisan,et al.  Fair allocation without trade , 2012, AAMAS.

[9]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[10]  Joseph Naor,et al.  A Truthful Mechanism for Value-Based Scheduling in Cloud Computing , 2011, SAGT.

[11]  Baochun Li,et al.  Dominant resource fairness in cloud computing systems with heterogeneous servers , 2013, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[12]  Mung Chiang,et al.  Multiresource allocation: fairness-efficiency tradeoffs in a unifying framework , 2013, TNET.