Capacity planning and scheduling in Grid computing environments

Grid computing infrastructures embody a cost-effective computing paradigm that virtualises heterogeneous system resources to meet the dynamic needs of critical business and scientific applications. These applications range from batch processes and long-running tasks to real-time and even transactional applications. Grid computing environments are inherently dynamic and unpredictable environments sharing services amongst many different users. Grid schedulers aim to make the most efficient use of Grid resources (high utilisation) while providing the best possible performance to the Grid applications (reducing makespan) and satisfying the associated performance and Quality of Service (QoS) constraints. Additionally, in commercial Grid settings where economic considerations are an increasingly important part of Grid scheduling, it is necessary to minimise the cost of application execution on the behalf of the Grid users while ensuring that the applications meet their QoS constraints. Furthermore, efficient resource allocation may allow a resource broker to maximise their profit by minimising the quantity of resource procurement. Scheduling in such a large-scale, dynamic and distributed environment is a complex undertaking. In this paper, we propose an approach to Grid scheduling which abstracts over the details of individual applications, focusing instead on the global cost optimisation problem while taking into account the entire workload, dynamically adjusting to the varying service demands. Our model places particular emphasis on the stochastic and unpredictable nature of the Grid, leading to a more accurate reflection of the state of the Grid and hence more efficient and accurate scheduling decisions.

[1]  Daniel A. Reed,et al.  Performance Contracts: Predicting and Monitoring Grid Application Behavior , 2001, GRID.

[2]  Yolanda Gil,et al.  Pegasus: Mapping Scientific Workflows onto the Grid , 2004, European Across Grids Conference.

[3]  Warren Smith,et al.  Scheduling with advanced reservations , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[4]  Francine Berman,et al.  A Decoupled Scheduling Approach for the GrADS Program Development Environment , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[5]  Satoshi Matsuoka,et al.  Performance Evaluation Model for Scheduling in Global Computing Systems , 2000, Int. J. High Perform. Comput. Appl..

[6]  Demetres D. Kouvatsos,et al.  Performance modelling and evaluation of heterogeneous networks , 2005, Perform. Evaluation.

[7]  Paul Allen,et al.  Service Orientation: Winning Strategies and Best Practices , 2006 .

[8]  Frederick S. Hillier,et al.  Introduction to Operations Research and Revised CD-ROM 8 , 2005 .

[9]  Lingyun Yang,et al.  Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[10]  Graham R. Nudd,et al.  Performance evaluation of an agent-based resource management infrastructure for grid computing , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[11]  Anurag Kumar,et al.  Adaptive Optimal Load Balancing in a Nonhomogeneous Multiserver System with a Central Job Scheduler , 1990, IEEE Trans. Computers.

[12]  J.M. Schopf,et al.  Stochastic Scheduling , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[13]  A. Gilles,et al.  The Art of Computer Systems Performance Analysis (Techniques for Experimental Design, Measurement, Simulation, and Modeling) , 1992 .

[14]  Steven Tuecke,et al.  The Physiology of the Grid An Open Grid Services Architecture for Distributed Systems Integration , 2002 .

[15]  Rajkumar Buyya,et al.  Economic-based Distributed Resource Management and Scheduling for Grid Computing , 2002, ArXiv.

[16]  Ali Afzal,et al.  Capacity Planning and Stochastic Scheduling in Large-Scale Grids , 2006, 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06).

[17]  Ali Afzal,et al.  Making the Grid Predictable through Reservations and Performance Modelling , 2005, Comput. J..

[18]  Daniel Rueckert,et al.  Performance prediction for a code with data‐dependent runtimes , 2008, Concurr. Comput. Pract. Exp..

[19]  Warren Smith,et al.  Predicting Application Run Times Using Historical Information , 1998, JSSPP.

[20]  Heather Fry,et al.  A user’s guide , 2003 .

[21]  Jack J. Dongarra,et al.  Scheduling workflow applications on processors with different capabilities , 2006, Future Gener. Comput. Syst..

[22]  Paul Avery,et al.  Data Grids: a new computational infrastructure for data-intensive science , 2002, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[23]  Hui Li,et al.  Efficient response time predictions by exploiting application and resource state similarities , 2005, The 6th IEEE/ACM International Workshop on Grid Computing, 2005..

[24]  Klara Nahrstedt,et al.  A distributed resource management architecture that supports advance reservations and co-allocation , 1999, 1999 Seventh International Workshop on Quality of Service. IWQoS'99. (Cat. No.98EX354).

[25]  Subhash Saini,et al.  Performance prediction and its use in parallel and distributed computing systems , 2006, Future Gener. Comput. Syst..

[26]  Rajkumar Buyya,et al.  GridSim: A Toolkit for the Modeling and Simulation of Global Grids , 2001 .

[27]  Grzegorz Waligóra,et al.  A metaheuristic approach to scheduling workflow jobs on a Grid , 2004 .

[28]  Isi Mitrani,et al.  Dynamic Server Allocation in Heterogeneous Clusters , 2003 .

[29]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[30]  Fan Zhang,et al.  Characterizing Normal Operation of a Web Server: Application to Workload Forecasting and Problem Determination , 1998, Int. CMG Conference.

[31]  Muthucumaru Maheswaran,et al.  Scheduling Co-Reservations with Priorities in Grid Computing Systems , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[32]  C. Floudas.  Nonlinear and Mixed-Integer Optimization: Fundamentals and Applications , 1995 .

[33]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[34]  Frederick S. Hillier,et al.  Introduction of Operations Research , 1967 .

[35]  Ali Afzal,et al.  Performance Architecture within ICENI , 2004 .

[36]  Morteza Analoui,et al.  QoS-based scheduling of workflow applications on grids , 2007 .

[37]  Paul McKee,et al.  Dynamic Allocation of Servers in a Grid Hosting Environment , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[38]  Javier Jaén Martínez,et al.  Data Management in an International Data Grid Project , 2000, GRID.

[39]  C. Tham,et al.  QoS-based Scheduling of Workflow Applications on Service Grids , 2005 .

[40]  Jennifer M. Schopf,et al.  Ten actions when Grid scheduling: the user as a Grid scheduler , 2004 .

[41]  Daniel Rueckert,et al.  Performance prediction for a code with data-dependent runtimes , 2008 .

[42]  Xin Li,et al.  Prophesy: an infrastructure for analyzing and modeling the performance of parallel and distributed applications , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.