Benchmarks for grid computing: a review of ongoing efforts and future directions

Grid architectures are collections of computational and data storage resources linked by communication channels for shared use. It is important to deploy measurement methods so that Grid applications and architectures can evolve guided by scientific principles. Engineering pursuits need agreed upon metrics---a common language for communicating results, so that alternative implementations can be compared quantitatively. Users of systems need performance parameters that describe system capabilities so that they can develop and tune their applications. Architects need examples of how users will exercise their system to improve the design. The Grid community is building systems such as the TeraGrid [1] and The Informational Power Grid [2] while applications that can fully benefit from such systems are also being developed. We conclude that the time to develop and deploy sets of Grid benchmarks is now. This article reviews fundamental principles, early efforts, and benefits of Grid benchmarks to the study and design of Grids.

[1]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[2]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[3]  Ian T. Foster,et al.  A security architecture for computational grids , 1998, CCS '98.

[4]  Holly Dail,et al.  A Modular Framework for Adaptive Scheduling in Grid Application Development Environments , 2002 .

[5]  Ian T. Foster,et al.  The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets , 2000, J. Netw. Comput. Appl..

[6]  Francine Berman,et al.  The GrADS Project: Software Support for High-Level Grid Application Development , 2001, Int. J. High Perform. Comput. Appl..

[7]  William E. Johnston,et al.  Grids as production computing environments: the engineering aspects of NASA's Information Power Grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[8]  Andrew A. Chien,et al.  The MicroGrid: a Scientific Tool for Modeling Computational Grids , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[9]  Steven Tuecke,et al.  The Physiology of the Grid An Open Grid Services Architecture for Distributed Systems Integration , 2002 .

[10]  Michael A. Frumkin,et al.  NAS Grid Benchmarks: A Tool for Grid Space Exploration , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[11]  Peter A. Dinda,et al.  A Unified Relational Approach to Grid Information Services , 2003, ICDCS 2003.

[12]  Warren Smith,et al.  A Resource Management Architecture for Metacomputing Systems , 1998, JSSPP.

[13]  Julian Borrill.  MADCAP - The Microwave Anisotropy Dataset Computational Analysis Package , 1999 .

[14]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[15]  Sathish S. Vadhiyar,et al.  Numerical Libraries And The Grid: The GrADS Experiments With ScaLAPACK , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[16]  Ian T. Foster,et al.  Grid information services for distributed resource sharing , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[17]  Ian T. Foster,et al.  A Grid-Enabled MPI: Message Passing in Heterogeneous Distributed Computing Systems , 1998, Proceedings of the IEEE/ACM SC98 Conference.