Distributed resource discovery in large scale computing systems

There has been significant effort to build high throughput delivering computing systems out of distributed workstations. These systems are growing to accommodate larger number of workstations with growing demand. Discovery of available resources in such environments is a challenging problem. We present a completely distributed resource discovery solution, which utilizes P2P design to provide a scalable service. Our design allows jobs to search for desired workstations, as well as, workstations to search for jobs that may run on them.

[1]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[2]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[3]  Miron Livny,et al.  Harnessing the Capacity of Computational Grids for High Energy Physics , 2000 .

[4]  Dejan S. Milojicic,et al.  Process migration , 1999, ACM Comput. Surv..

[5]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[6]  Ian Foster,et al.  On Fully Decentralized Resource Discovery in Grid Environments , 2001, GRID.

[7]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[8]  Ian T. Foster,et al.  The Anatomy of the Grid: Enabling Scalable Virtual Organizations , 2001, Int. J. High Perform. Comput. Appl..

[9]  Rajesh Raman,et al.  Policy driven heterogeneous resource co-allocation with Gangmatching , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[10]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[11]  Divyakant Agrawal,et al.  A peer-to-peer framework for caching range queries , 2004, Proceedings. 20th International Conference on Data Engineering.

[12]  Ian T. Foster,et al.  Condor-G: A Computation Management Agent for Multi-Institutional Grids , 2004, Cluster Computing.