Meridian: a lightweight network location service without virtual coordinates

This paper introduces a lightweight, scalable and accurate framework, called Meridian, for performing node selection based on network location. The framework consists of an overlay network structured around multi-resolution rings, query routing with direct measurements, and gossip protocols for dissemination. We show how this framework can be used to address three commonly encountered problems, namely, closest node discovery, central leader election, and locating nodes that satisfy target latency constraints in large-scale distributed systems without having to compute absolute coordinates. We show analytically that the framework is scalable with logarithmic convergence when Internet latencies are modeled as a growth-constrained metric, a low-dimensional Euclidean metric, or a metric of low doubling dimension. Large scale simulations, based on latency measurements from 6.25 million node-pairs as well as an implementation deployed on PlanetLab show that the framework is accurate and effective.

[1]  R. Varga,et al.  Proof of Theorem 4 , 1983 .

[2]  Scott Shenker,et al.  Epidemic algorithms for replicated database maintenance , 1988, OPSR.

[3]  Michael F. Schwartz,et al.  Locating nearby copies of replicated Internet servers , 1995, SIGCOMM '95.

[4]  Rajeev Motwani,et al.  Randomized algorithms , 1996, CSUR.

[5]  Rajmohan Rajaraman,et al.  Accessing Nearby Copies of Replicated Objects in a Distributed Environment , 1997, SPAA '97.

[6]  Mark Crovella,et al.  Server selection using dynamic path characterization in wide-area networks , 1997, Proceedings of INFOCOM '97.

[7]  Ellen W. Zegura,et al.  A novel server selection technique for improving the response time of a replicated service , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[8]  Stefan Savage,et al.  The end-to-end effects of Internet path selection , 1999, SIGCOMM '99.

[9]  Mark Crovella,et al.  On the network impact of dynamic server selection , 1999, Comput. Networks.

[10]  Hui Zhang,et al.  A case for end system multicast (keynote address) , 2000, SIGMETRICS '00.

[11]  Paul Francis,et al.  IDMaps: a global internet host distance estimation service , 2001, TNET.

[12]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[13]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[14]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[15]  Bobby Bhattacharjee,et al.  Finding close friends on the Internet , 2001, Proceedings Ninth International Conference on Network Protocols. ICNP 2001.

[16]  Jon M. Kleinberg,et al.  Spatial gossip and resource location protocols , 2001, JACM.

[17]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[18]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[19]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[20]  Kirk L. Johnson,et al.  The measured performance of content distribution networks , 2001, Comput. Commun..

[21]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[22]  Ben Y. Zhao,et al.  Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and , 2001 .

[23]  Mark Handley,et al.  Topologically-aware overlay construction and server selection , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[24]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[25]  Randy H. Katz,et al.  Backup path allocation based on a correlated link failure probability model in overlay networks , 2002, 10th IEEE International Conference on Network Protocols, 2002. Proceedings..

[26]  Srinivasan Seshan,et al.  A case for end system multicast , 2002, IEEE J. Sel. Areas Commun..

[27]  Bobby Bhattacharjee,et al.  Scalable peer finding on the Internet , 2002, Global Telecommunications Conference, 2002. GLOBECOM '02. IEEE.

[28]  David R. Karger,et al.  Finding nearest neighbors in growth-restricted metrics , 2002, STOC '02.

[29]  Ben Y. Zhao,et al.  Distributed Object Location in a Dynamic Network , 2002, SPAA '02.

[30]  Peter Druschel,et al.  Exploiting network proximity in peer-to-peer overlay networks , 2002 .

[31]  Krishna P. Gummadi,et al.  King: estimating latency between arbitrary internet end hosts , 2002, IMW '02.

[32]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[33]  John Kubiatowicz,et al.  Introspective failure analysis: avoiding correlated failures in peer-to-peer systems , 2002, 21st IEEE Symposium on Reliable Distributed Systems, 2002. Proceedings..

[34]  Marcel Waldvogel,et al.  Efficient topology-aware overlay network , 2003, CCRV.

[35]  Kirsten Hildrum,et al.  Another Way to Find the Nearest Neighbor in Growth-Restricted Metrics , 2003 .

[36]  Mark Crovella,et al.  Virtual landmarks for the internet , 2003, IMC '03.

[37]  Robert Krauthgamer,et al.  Bounded geometries, fractals, and low-distortion embeddings , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[38]  Jon Crowcroft,et al.  Lighthouses for Scalable Distributed Location , 2003, IPTPS.

[39]  Peter Druschel,et al.  Proximity Neighbor Selection in Tree-Based Structured Peer-to-Peer Overlays , 2003 .

[40]  Mary Baker,et al.  Preserving peer replicas by rate-limited sampled voting , 2003, SOSP '03.

[41]  Robert Krauthgamer,et al.  The intrinsic dimensionality of graphs , 2003, STOC '03.

[42]  P. Assouad.  Plongements lipschitziens dans Rn , 2003 .

[43]  Hui Zhang,et al.  A Network Positioning System for the Internet , 2004, USENIX Annual Technical Conference, General Track.

[44]  Miguel Castro,et al.  PIC: practical Internet coordinates for distance estimation , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[45]  Robert Krauthgamer,et al.  Navigating nets: simple algorithms for proximity search , 2004, SODA '04.

[46]  Robert Krauthgamer,et al.  The Black-Box Complexity of Nearest Neighbor Search , 2004, ICALP.

[47]  Yuval Shavitt,et al.  Big-bang simulation for embedding network distances in Euclidean space , 2004, IEEE/ACM Transactions on Networking.

[48]  Kunal Talwar.  Bypassing the Embedding: Approximation schemes and Compact Representations for Low Dimensional Metrics , 2004 .

[49]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[50]  Srinivasan Seshan,et al.  Mercury: supporting scalable multi-attribute range queries , 2004, SIGCOMM '04.

[51]  Richard Lawrence.  Running Massively Multiplayer Games as a Business , 2004, NSDI.

[52]  Satish Rao,et al.  A note on the nearest neighbor in growth-restricted metrics , 2004, SODA '04.

[53]  Kunal Talwar,et al.  Bypassing the embedding: algorithms for low dimensional metrics , 2004, STOC '04.

[54]  David E. Culler,et al.  Operating Systems Support for Planetary-Scale Network Services , 2004, NSDI.

[55]  Robert Krauthgamer,et al.  Measured descent: a new embedding method for finite metrics , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[56]  Robert Krauthgamer,et al.  Measured Descent: A New Embedding Method for Finite Metrics , 2004, FOCS.

[57]  Robert Krauthgamer,et al.  Object location in realistic networks , 2004, SPAA '04.

[58]  Jon M. Kleinberg,et al.  Triangulation and embedding using small sets of beacons , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[59]  Steven Lerman,et al.  PCoord: network position estimation using peer-to-peer measurements , 2004, Third IEEE International Symposium on Network Computing and Applications, 2004. (NCA 2004). Proceedings..

[60]  Johannes Gehrke,et al.  Querying peer-to-peer networks using P-trees , 2004, WebDB '04.

[61]  Michael J. Freedman,et al.  Locality Prediction for Oblivious Clients , 2005, IPTPS.

[62]  Hyuk Lim,et al.  Constructing Internet coordinate system based on delay measurement , 2003, IEEE/ACM Transactions on Networking.

[63]  Aleksandrs Slivkins,et al.  Distributed approaches to triangulation and embedding , 2005, SODA '05.

[64]  Meridian: a lightweight network location service without virtual coordinates , 2005, SIGCOMM.

[65]  Aleksandrs Slivkins.  Distance estimation and object location via rings of neighbors , 2006, Distributed Computing.