Challenges in executing large parameter sweep studies across widely distributed computing environments

Large parameter sweep studies are common in a broad range of scientific disciplines. However, many challenges exist in supporting this type of computation in a widely distributed computing environment. These challenges exist because contributing sites in a federated distributed computing environment usually expose only a very narrow resource-sharing interface. This paper looks at the challenges encountered by parameter sweep studies using two concrete application examples. The paper also shows how a system for building personal clusters on demand has been developed to solve many of these problems.

[1]  Charles H. Bennett,et al.  Efficient estimation of free energy differences from Monte Carlo data , 1976 .

[2]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[3]  Thomas E. Cheatham,et al.  The flying ice cube: Velocity rescaling in molecular dynamics leads to violation of energy equipartition , 1998, J. Comput. Chem..

[4]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[5]  Edward Walker,et al.  A Distributed File System for a Wide-Area High Performance Computing Infrastructure , 2006, WORLDS.

[6]  William L. Jorgensen,et al.  Efficient computation of absolute free energies of binding by computer simulations. Application to the methane dimer in water , 1988 .

[7]  Pengyu Y. Ren,et al.  Polarizable Atomic Multipole Water Model for Molecular Mechanics Simulation , 2003 .

[8]  Edward Walker,et al.  Creating personal adaptive clusters for managing scientific jobs in a distributed computing environment , 2006, 2006 IEEE Challenges of Large Applications in Distributed Environments.

[9]  Edward Walker,et al.  Personal adaptive clusters as containers for scientific jobs , 2007, Cluster Computing.

[10]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[11]  Michael W. Deem,et al.  Toward a Database of Hypothetical Zeolite Structures , 2006 .