Scheduling DAG Applications for Time Sharing Systems

When computing the makespan of a DAG, it is typically assumed that the tasks scheduled on the same computing node run in sequence. In reality, however, the tasks may be run in the time sharing manner. Our studies show that the discrepancy between the assumption of sequential execution and the reality of time sharing execution may lead to inaccurate calculation of the DAG makespan. In this paper, we first investigate the impact of the time sharing execution on the DAG makespan, and propose the method to model and determine the makespan with the time-sharing execution. Based on this model, we further develop the scheduling strategies for DAG jobs running in time-sharing. Extensive experiments have been conducted to verify the effectiveness of the proposed methods. The experimental results show that by taking time sharing into account, our DAG scheduling strategy can reduce the makespan significantly, comparing with its counterpart in sequential execution.

[1]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[2]  Srikanth Kandula,et al.  Efficient queue management for cluster scheduling , 2016, EuroSys.

[3]  Carlo Curino,et al.  Mercury: Hybrid Centralized and Distributed Scheduling in Large Shared Clusters , 2015, USENIX Annual Technical Conference.

[4]  Stephen A. Jarvis,et al.  Developing communication-aware service placement frameworks in the Cloud economy , 2013, 2013 IEEE International Conference on Cluster Computing (CLUSTER).

[5]  Xiao Zhang,et al.  CPI2: CPU performance isolation for shared compute clusters , 2013, EuroSys '13.

[6]  Yibin Li,et al.  An Improved Energy-Efficient Scheduling for Precedence Constrained Tasks in Multiprocessor Clusters , 2014, ICA3PP.

[7]  Anthony A. Maciejewski,et al.  Makespan and Energy Robust Stochastic Static Resource Allocation of a Bag-of-Tasks to a Heterogeneous Computing System , 2015, IEEE Transactions on Parallel and Distributed Systems.

[8]  Albert Y. Zomaya,et al.  Energy-aware parallel task scheduling in a cluster , 2013, Future Gener. Comput. Syst..

[9]  Patrick Wendell,et al.  Sparrow: distributed, low latency scheduling , 2013, SOSP.

[10]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..

[11]  Wei Lin,et al.  Apollo: Scalable and Coordinated Scheduling for Cloud-Scale Computing , 2014, OSDI.

[12]  Tao Li,et al.  Scheduling Stochastic Tasks with Precedence Constrain on Cluster Systems with Heterogenous Communication Architecture , 2015, ICA3PP.

[13]  Ying Zhang,et al.  DCloud: Deadline-Aware Resource Allocation for Cloud Computing Jobs , 2016, IEEE Transactions on Parallel and Distributed Systems.

[14]  Abhishek Verma,et al.  Large-scale cluster management at Google with Borg , 2015, EuroSys.

[15]  Venkatram Vishwanath,et al.  Workflow performance improvement using model-based scheduling over multiple clusters and clouds , 2016, Future Gener. Comput. Syst..

[16]  Rajkumar Buyya,et al.  Deadline‐constrained coevolutionary genetic algorithm for scientific workflow scheduling in cloud computing , 2017, Concurr. Comput. Pract. Exp..

[17]  Hao Wu,et al.  Resource and Instance Hour Minimization for Deadline Constrained DAG Applications Using Computer Clouds , 2016, IEEE Transactions on Parallel and Distributed Systems.

[18]  Keqin Li,et al.  Schedule length minimization of parallel applications with energy consumption constraints using heuristics on heterogeneous distributed systems , 2017, Concurr. Comput. Pract. Exp..

[19]  Laurent Philippe,et al.  On the Heterogeneity Bias of Cost Matrices for Assessing Scheduling Algorithms , 2017, IEEE Trans. Parallel Distributed Syst..

[20]  Keqin Li,et al.  Efficient task scheduling for budget constrained parallel applications on heterogeneous cloud computing systems , 2017, Future Gener. Comput. Syst..

[21]  Kenli Li,et al.  Slack allocation algorithm for energy minimization in cluster systems , 2017, Future Gener. Comput. Syst..

[22]  Carlo Curino,et al.  Apache Hadoop YARN: yet another resource negotiator , 2013, SoCC.