Rarity-Based Routing in Structured Peer-to-Peer Overlays

The OpenKnowledge project aims at knowledge sharing through open and flexible peer interactions. Within this project, we are developing a system that supports searching, developing and sharing of interactions/workflows consisting of roles implemented by software that can be shared and executed by peers. Its main requirements are openness, scalability, decentralization and robustness. Part of this system is a discovery service, which will be the focus of this paper. This service aspires to fulfill the above requirements featuring a Peer-to-Peer architecture and Distributed Hash Tables (DHTs) to achieve robustness through redundancy and scalability through decentralization. Resources are discovered using a set of attribute-value pairs. A straightforward DHT-based approach that creates a distributed inverted index suffers from a linear increase of messages and replicas with the number of attributes. We try to reduce this number by proposing an efficient multi-attribute routing algorithm. We emulate and test our implementation on the DAS-2 distributed supercomputer.

[1]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[2]  Gurmeet Singh Manku,et al.  Symphony: Distributed Hashing in a Small World , 2003, USENIX Symposium on Internet Technologies and Systems.

[3]  Antonio F. Gómez-Skarmeta,et al.  Cyclone: a novel design schema for hierarchical DHTs , 2005, Fifth IEEE International Conference on Peer-to-Peer Computing (P2P'05).

[4]  Karl Aberer,et al.  P-Grid: a self-organizing structured P2P system , 2003, SGMD.

[5]  David Stuart Robertson,et al.  Multi-agent Coordination as Distributed Logic Programming , 2004, ICLP.

[6]  Karl Aberer,et al.  Distributed cache table: efficient query-driven processing of multi-term queries in P2P networks , 2006, P2PIR '06.

[7]  Srinivasan Seshan,et al.  Mercury: supporting scalable multi-attribute range queries , 2004, SIGCOMM '04.

[8]  Ronny Siebes.  pNear: combining Content Clustering and Distributed Hash Tables , 2005, P2PKM.

[9]  D. Dupplaw,et al.  The Open Knowledge Kernel , 2007 .

[10]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[11]  Ben Y. Zhao,et al.  Towards a Common API for Structured Peer-to-Peer Overlays , 2003, IPTPS.

[12]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[13]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[14]  Bruce M. Maggs,et al.  Efficient content location using interest-based locality in peer-to-peer systems , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[15]  Ion Stoica,et al.  The Case for a Hybrid P2P Search Infrastructure , 2004, IPTPS.