A Distributed Architecture for Multi-dimensional Indexing and Data Retrieval in Grid Environments ∗

In this paper, we describe a service-oriented architecture of a generic middleware platform, which provides the required services for data management in a distributed environment. Our design introduces concepts from Peer-to-Peer computing in order to provide a scalable and reliable infrastructure for storage, search and retrieval of annotated content. To ensure fast searching in the distributed repositories of a Virtual Organization, our system incorporates a multidimensional indexing scheme, which serves the need for supporting both point and range queries over a set of metadata attributes. Finally, multimedia file transfers are conducted using GridTorrent, a grid-enabled, Peer-toPeer mechanism that allows the aggregate data transfer throughput to scale and effectively copes with flash crowds.

[1]  Carl Kesselman,et al.  Grid-based metadata services , 2004, Proceedings. 16th International Conference on Scientific and Statistical Database Management, 2004..

[2]  Peter Z. Kunszt,et al.  Giggle: A Framework for Constructing Scalable Replica Location Services , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[3]  Ian T. Foster,et al.  Globus Toolkit Version 4: Software for Service-Oriented Systems , 2005, Journal of Computer Science and Technology.

[4]  Ian T. Foster,et al.  The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets , 2000, J. Netw. Comput. Appl..

[5]  Ian T. Foster,et al.  A security architecture for computational grids , 1998, CCS '98.

[6]  Nectarios Koziris,et al.  A peer-to-peer replica management service for high-throughput grids , 2005, 2005 International Conference on Parallel Processing (ICPP'05).

[7]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[8]  Carl Kesselman,et al.  Performance and scalability of a replica location service , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[9]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[10]  K. Doka,et al.  GridTorrent : Optimizing data transfers in the Grid with collaborative sharing , 2006 .