A peer-to-peer replica management service for high-throughput grids

Future high-throughput grids may integrate millions or even billions of processing and data storage nodes. Services provided by the underlying grid infrastructure may have to be able to scale to capacities not even imaginable today. In this paper we concentrate on one of the core components of the data grid architecture - the replica location service - and evaluate a redesign of the system based on a structured peer-to-peer network overlay. We argue that the architecture of the currently most widespread solution for file replica location on the grid is biased towards high-performance deployments and cannot scale to the future needs of a global grid. Structured peer-to-peer systems can provide the same functionality, while being much more manageable, scalable and fault-tolerant. However, they are only capable of storing read-only data. To this end, we propose a revised protocol for distributed hash tables that allows data to be changed in a distributed and scalable fashion. Results from a prototype implementation of the system suggest that grids can truly benefit from the scalability and fault-tolerance properties of such peer-to-peer algorithms.

[1]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[2]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[3]  Javier Jaén Martínez,et al.  Data Management in an International Data Grid Project , 2000, GRID.

[4]  Andrew Hanushevsky,et al.  HTTP redirection for replica catalogue lookups in data grids , 2002, SAC '02.

[5]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[6]  Robert Tappan Morris,et al.  Ivy: a read/write peer-to-peer file system , 2002, OSDI '02.

[7]  Ian T. Foster,et al.  Mapping the Gnutella Network: Macroscopic Properties of Large-Scale Peer-to-Peer Systems , 2002, IPTPS.

[8]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[9]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[10]  Klemens Böhm,et al.  How to run experiments with large peer-to-peer data structures , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[11]  Peter Z. Kunszt,et al.  Giggle: A Framework for Constructing Scalable Replica Location Services , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[12]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[13]  Ian T. Foster,et al.  A peer-to-peer approach to resource location in grid environments , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[14]  Min Cai,et al.  A Peer-to-Peer Replica Location Service Based on a Distributed Hash Table , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[15]  Federico Ruggieri.  The Datagrid Project , 2001 .

[16]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[17]  Ian T. Foster,et al.  The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets , 2000, J. Netw. Comput. Appl..

[18]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[19]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[20]  Ian T. Foster,et al.  On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing , 2003, IPTPS.

[21]  Carl Kesselman,et al.  Performance and scalability of a replica location service , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[22]  Robert Tappan Morris,et al.  Comparing the Performance of Distributed Hash Tables Under Churn , 2004, IPTPS.

[23]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[24]  Ian Foster,et al.  A Decentralized, Adaptive, Replica Location Service , 2002 .

[25]  David E. Culler,et al.  A blueprint for introducing disruptive technology into the Internet , 2003, CCRV.

[26]  Ian T. Foster,et al.  Grid information services for distributed resource sharing , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[27]  Lada A. Adamic,et al.  Search in Power-Law Networks , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  Thomer M. Gil,et al.  Ivy: a read/write peer-to-peer file system , 2002, OSDI '02.

[29]  David R. Karger,et al.  Looking up data in P2P systems , 2003, CACM.

[30]  Erwin Laure,et al.  Advanced Replica Management with Reptor , 2003, PPAM.

[31]  Ian Foster,et al.  A peer-to-peer approach to resource location in grid environments , 2002 .

[32]  Steven Hazel,et al.  Achord: A Variant of the Chord Lookup Service for Use in Censorship Resistant Peer-to-Peer Publishing Systems , 2002 .

[33]  Brian Tierney,et al.  File and Object Replication in Data Grids , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.