A grid middleware for data management exploiting peer-to-peer techniques

In this paper, we describe a service-oriented middleware architecture for Grid environments which enables efficient data management. Our design introduces concepts from Peer-to-Peer computing in order to provide a scalable and reliable infrastructure for storage, search and retrieval of annotated content. To ensure fast file lookups in the distributed repositories, our system incorporates a multidimensional indexing scheme which serves the need for supporting both exact match and range queries over a group of metadata attributes. Finally, file transfers are conducted using GridTorrent, a grid-enabled, Peer-to-Peer mechanism that performs efficient data transfers by enabling cooperation among participating nodes and balances the cost of file transfer among them. The proposed architecture is the middleware component used by the GREDIA project, in which both media and banking partners plan to share large loads of annotated content.

[1]  Steven Tuecke,et al.  The Open Grid Services Architecture , 2004, The Grid 2, 2nd Edition.

[2]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[3]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[4]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[5]  Manish Parashar,et al.  Enabling flexible queries with guarantees in P2P systems , 2004, IEEE Internet Computing.

[6]  Su Myeon Kim,et al.  CISS: An efficient object clustering framework for DHT-based peer-to-peer applications , 2004, Comput. Networks.

[7]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[8]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[9]  Ian T. Foster,et al.  A security architecture for computational grids , 1998, CCS '98.

[10]  Arthur R. Butz,et al.  Alternative Algorithm for Hilbert's Space-Filling Curve , 1971, IEEE Transactions on Computers.

[11]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[12]  Nectarios Koziris,et al.  A peer-to-peer replica management service for high-throughput grids , 2005, 2005 International Conference on Parallel Processing (ICPP'05).

[13]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM 2001.

[14]  Carl Kesselman,et al.  Grid-based metadata services , 2004, Proceedings. 16th International Conference on Scientific and Statistical Database Management, 2004..

[15]  Hector Garcia-Molina,et al.  One torus to rule them all: multi-dimensional queries in P2P systems , 2004, WebDB '04.

[16]  Mike Jackson,et al.  Introduction to OGSA-DAI Services , 2004, SAG.

[17]  Eddy Caron,et al.  Enhancing Computational Grids with Peer-to-Peer Technology for Large Scale Service Discovery , 2007, Journal of Grid Computing.

[18]  Jie Liu,et al.  A scalable P2P platform for the knowledge grid , 2005, IEEE Transactions on Knowledge and Data Engineering.

[19]  Ian T. Foster,et al.  Data management and transfer in high-performance computational grid environments , 2002, Parallel Comput..

[20]  Ian Clarke,et al.  Freenet: A Distributed Anonymous Information Storage and Retrieval System , 2000, Workshop on Design Issues in Anonymity and Unobservability.

[21]  T. Mexia,et al.  Author ' s personal copy , 2009 .

[22]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[23]  Karl Aberer,et al.  Range queries in trie-structured overlays , 2005, Fifth IEEE International Conference on Peer-to-Peer Computing (P2P'05).

[24]  Sriram Ramabhadran,et al.  Brief announcement: prefix hash tree , 2004, PODC '04.

[25]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[26]  Ian T. Foster,et al.  The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets , 2000, J. Netw. Comput. Appl..

[27]  Walid G. Aref,et al.  Analysis of Multi-Dimensional Space-Filling Curves , 2003, GeoInformatica.

[28]  Ian T. Foster,et al.  Globus Toolkit Version 4: Software for Service-Oriented Systems , 2005, Journal of Computer Science and Technology.

[29]  Christos Faloutsos,et al.  Analysis of the Clustering Properties of the Hilbert Space-Filling Curve , 2001, IEEE Trans. Knowl. Data Eng..

[30]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[31]  Peter Z. Kunszt,et al.  Giggle: A Framework for Constructing Scalable Replica Location Services , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[32]  Carl Kesselman,et al.  Performance and scalability of a replica location service , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..