Adlib: a self-tuning index for dynamic peer-to-peer systems

Peer-to-peer (P2P) systems enable queries over a large database horizontally partitioned across a dynamic set of nodes. We devise a self-tuning index for such systems that can trade off index maintenance cost against query efficiency, in order to optimize the overall system cost. The index, Adlib, dynamically adapts itself to operate at the optimal trade-off point, even as the optimal configuration changes with nodes joining and leaving the system. We use experiments on realistic workloads to demonstrate that Adlib can reduce the overall system cost by a factor of four.

[1]  Gurmeet Singh Manku,et al.  Routing networks for distributed hash tables , 2003, PODC '03.

[2]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[3]  Krishna P. Gummadi,et al.  Canon in G major: designing DHTs with hierarchical structure , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[4]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[5]  Scott Shenker,et al.  Querying the Internet with PIER , 2003, VLDB.

[6]  Hector Garcia-Molina,et al.  YAPPERS: a peer-to-peer lookup service over arbitrary topology , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[7]  Karl Aberer,et al.  P-Grid: A Self-Organizing Access Structure for P2P Information Systems , 2001, CoopIS.

[8]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[9]  Hector Garcia-Molina,et al.  Online Balancing of Range-Partitioned Data with Applications to Peer-to-Peer Systems , 2004, VLDB.

[10]  Hector Garcia-Molina,et al.  Comparing Hybrid Peer-to-Peer Systems , 2001, VLDB.

[11]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[12]  Mark Handley,et al.  Topologically-aware overlay construction and server selection , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[13]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[14]  Ben Y. Zhao,et al.  Distributed Object Location in a Dynamic Network , 2002, SPAA '02.

[15]  Hector Garcia-Molina,et al.  Adlib: A Self-Tuning Index for Dynamic P2P Systems. , 2005, ICDE 2005.

[16]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[17]  Hector Garcia-Molina,et al.  Improving Search in Peer-to-Peer Systems , 2001 .

[18]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[19]  Forouzan Golshani,et al.  Proceedings of the Eighth International Conference on Data Engineering , 1992 .