The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid

The Computational Grid is a promising platform for the efficient execution of parameter sweep applications over large parameter spaces. To achieve performance on the Grid, such applications must be scheduled so that shared data files are strategically placed to maximize re-use, and so that the application execution can adapt to the deliverable performance potential of target heterogeneous, distributed and shared resources. Parameter sweep applications are an important class of applications and would greatly benefit from the development of Grid middleware that embeds a scheduler for performance and targets Grid resources transparently. In this paper we describe a user-level Grid middleware project, the AppLeS Parameter Sweep Template (APST), that uses application-level scheduling techniques [1] and various Grid technologies to allow the efficient deployment of parameter sweep applications over the Grid. We discuss several possible scheduling algorithms and detail our software design. We then describe our current implementation of APST using systems like Globus [2], NetSolve [3] and the Network Weather Service [4], and present experimental results.

[1]  Joel R. Stiles,et al.  Monte Carlo simulation of neuro-transmitter release using MCell, a general simulator of cellular physiological processes , 1998 .

[2]  R. Wolski,et al.  Predicting the CPU availability of time‐shared Unix systems on the computational grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[3]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[4]  D. Rogers,et al.  EGS4 code system , 1985 .

[5]  Francine Berman,et al.  The AppLeS Project: A Status Report , 1997 .

[6]  David Abramson,et al.  High performance parametric modeling with Nimrod/G: killer application for the global grid? , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[7]  S Rogers,et al.  A comparison of implicit schemes for the incompressible Navier-Stokes equations with artificial compressibility , 1995 .

[8]  Andrew S. Grimshaw,et al.  Wide-Area Computing: Resource Sharing on a Large Scale , 1999, Computer.

[9]  Michelle Miller,et al.  An integrated problem solving environment: the SCIRun computational steering system , 1998, Proceedings of the Thirty-First Hawaii International Conference on System Sciences.

[10]  R. F. Freund,et al.  Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[11]  Mitsuhisa Sato,et al.  Resource manager for Globus-based wide-area cluster computing , 1999, ICWC 99. IEEE Computer Society International Workshop on Cluster Computing.

[12]  Michael Pinedo,et al.  Scheduling: Theory, Algorithms, and Systems , 1994 .

[13]  Ami Marowka,et al.  What is the GRID? , 2002, Scalable Comput. Pract. Exp..

[14]  Francine Berman,et al.  Application-Level Scheduling on Distributed Heterogeneous Networks , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[15]  Richard Wolski,et al.  Dynamically forecasting network performance using the Network Weather Service , 1998, Cluster Computing.

[16]  Peter Arbenz,et al.  The Remote Computation System , 1996, Parallel Comput..

[17]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..

[18]  Micah Beck,et al.  The Internet Backplane Protocol: Storage in the Network , 1999 .

[19]  Warren Smith,et al.  A Resource Management Architecture for Metacomputing Systems , 1998, JSSPP.

[20]  Ladislau Bölöni,et al.  A comparison study of static mapping heuristics for a class of meta-tasks on heterogeneous computing systems , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[21]  Francine Berman,et al.  Heuristics for scheduling parameter sweep applications in grid environments , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[22]  David Abramson,et al.  Modelling Photochemical Pollution using Parallel and Distributed Computing Platforms , 1994, PARLE.

[23]  Michael Mitzenmacher,et al.  How Useful Is Old Information? , 2000, IEEE Trans. Parallel Distributed Syst..

[24]  Ian T. Foster,et al.  GASS: a data movement and access service for wide area computing systems , 1999, IOPADS '99.

[25]  Amitava Majumdar.  Parallel performance study of Monte Carlo photon transport code on shared-, distributed-, and distributed-shared-memory architectures , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[26]  Torben Hagerup.  Allocating Independent Tasks to Parallel Processors: An Experimental Study , 1996, IRREGULAR.

[27]  Henry Wallace Clark,et al.  The Gantt Chart , 1947 .

[28]  Oscar H. Ibarra,et al.  Heuristic Algorithms for Scheduling Independent Tasks on Nonidentical Processors , 1977, JACM.

[29]  Miron Livny,et al.  Harnessing the Capacity of Computational Grids for High Energy Physics , 2000 .

[30]  Rajesh Raman,et al.  High Throughput Monte Carlo , 1999, PPSC.

[31]  Dean Sutherland,et al.  A resource query interface for network-aware applications , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[32]  Francine Berman,et al.  Using Effective Network Views to Promote Distributed Application Performance , 1999, PDPTA.