Dimensioning and on-line scheduling in Lambda Grids using divisible load concepts

Abstract Due to the large amounts of data required to be processed by the typical Grid job, it is conceivable that the use of optical transport networks in Grid deployment (hence the term “Lambda Grid”) will increase. The exact topology of the interconnecting network is obtained by solving a dimensioning problem, and the outcome of this strongly depends on both the expected workload characteristics and Grid scheduling policy. Solving this combined scheduling and dimensioning problem using straightforward ILP modelling is cumbersome; however, for steady-state Grid operation, Divisible Load Theory (DLT) can yield scalable formulations of this problem. In this paper, the on-line hierarchical scheduling on a lambda Grid of workload approaching the Grid’s capacity in a two-tier Grid mode of operation is studied. A number of these algorithms are goal-driven, in the sense that target per-resource goals are obtained from the off-line solution to the Divisible Load model. We compare these on-line multiresource scheduling policies for different workloads, Grid interconnection topologies and Grid parameters. We show that these algorithms perform well in the studied scenarios when compared to a fully centralized scheduling algorithm.

[1]  Anca I. D. Bucur,et al.  The Influence of the Structure and Sizes of Jobs on the Performance of Co-allocation , 2000, JSSPP.

[2]  Uwe Schwiegelshohn,et al.  Theory and Practice in Parallel Job Scheduling , 1997, JSSPP.

[3]  Cees T. A. M. de Laat,et al.  TransLight: a global-scale LambdaGrid for e-science , 2003, CACM.

[4]  Amnon Barak,et al.  Opportunity Cost Algorithms for Reduction of I/O and Interprocess Communication Overhead in a Computing Cluster , 2003, IEEE Trans. Parallel Distributed Syst..

[5]  Anca I. D. Bucur,et al.  An evaluation of processor co-allocation for different system configurations and job structures , 2002, 14th Symposium on Computer Architecture and High Performance Computing, 2002. Proceedings..

[6]  Kavitha Ranganathan,et al.  Simulation Studies of Computation and Data Scheduling Algorithms for Data Grids , 2003, Journal of Grid Computing.

[7]  Floriano Zini,et al.  Evaluating scheduling and replica optimisation strategies in OptorSim , 2003, Proceedings. First Latin American Web Congress.

[8]  Henri Casanova,et al.  A realistic network/application model for scheduling divisible loads on large-scale platforms , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[9]  Bruno Volckaert,et al.  Scalable dimensioning of optical transport networks for grid excess load handling , 2006, Photonic Network Communications.

[10]  L. Smarr,et al.  The OptIPuter, Quartzite, and Starlight projects: a campus to global-scale testbed for optical technologies enabling LambdaGrid computing , 2005, OFC/NFOEC Technical Digest. Optical Fiber Communication Conference, 2005..

[11]  Rema Padman,et al.  An integrated survey of deterministic project scheduling , 2001 .

[12]  Thomas G. Robertazzi,et al.  Divisible Load Scheduling for Grid Computing , 2003 .

[13]  S. Majumdar,et al.  Dynamic scheduling of lightpaths in lambda grids , 2005, 2nd International Conference on Broadband Networks, 2005..

[14]  Rema Padman,et al.  An integrated survey of project scheduling , 1997 .

[15]  Jirí Sgall,et al.  On-line Scheduling , 1996, Online Algorithms.

[16]  Achim Streit,et al.  Enhanced Algorithms for Multi-site Scheduling , 2002, GRID.

[17]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[18]  Jiri Sgall,et al.  On-line scheduling --- a survey , 1997 .

[19]  Bruno Volckaert,et al.  Network Aspects of Grid Scheduling Algorithms , 2004, ISCA PDCS.

[20]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[21]  Thomas G. Robertazzi,et al.  Scalable Scheduling in Parallel Processors , 2002 .

[22]  Rajkumar Buyya,et al.  A Deadline and Budget Constrained Cost-Time Optimisation Algorithm for Scheduling Task Farming Applications on Global Grids , 2002, ArXiv.

[23]  David B. Shmoys,et al.  Scheduling to Minimize Average Completion Time: Off-Line and On-Line Approximation Algorithms , 1997, Math. Oper. Res..