Algorithms for reliable peer-to-peer networks

This thesis addresses the problem of the lack of reliability in peer-to-peer networks, and proposes a number of algorithms that can provide reliability guarantees to peer-to-peer applications. Note that reliability in a peer-to-peer networking context is different from TCP type reliability. We define a reliable peer-to-peer as a network that is resilient to changes such as network dynamics, and can offer participating peers increased performance when possible. We make the following contributions to area of peer-to-peer reliability: (1) we propose an algorithm that creates resilient low-diameter topologies that guarantee an upper bound on delays among nodes; (2) we study parallel downloads in peer-to-peer networks and how they affect nodes by looking at their utilities and the overall performance of the network; and (3) we investigate network metrics relevant to peer-to-peer networks and their estimation using incomplete information. While we focus on latency and hop count as drivers for improving the performance of the peers, the proposed approach is more generally applicable to other network-wide metrics (e.g., bandwidth, loss). Our research methodology encompasses simulations and analytical analysis to understand the behavior and properties of the proposed systems, and substantial experimentation, as practical proof of concept of our ideas, using the PlanetLab platform. The common overarching theme of the thesis is the design of new resilient network algorithms capable of offering high-performance to peers and their applications. As more and more applications rely on underlying peer-to-peer topologies, the need for efficient and resilient infrastructure has become more pressing. A number of important classes of topologies have emerged over the last several years, all of which have various strengths and weaknesses. For example, the popular structured peer-to-peer topologies based on Distributed Hash Tables (DHTs) offer applications assured performance, but are not resilient to attacks and major disruptions that are likely in the overlay. In contrast, unstructured topologies where nodes create random connections among themselves on-the-fly, are resilient to attacks but can not offer performance assurances because they often create overlays with large diameters, making some nodes practically unreachable. In our first contribution, we propose Phenix, an algorithm for building resilient low-diameter peer-to-peer topologies that can resist different types of organized and targeted malicious behavior. (Abstract shortened by UMI.)

[1]  Sujata Banerjee,et al.  Netvigator: Scalable Network Proximity Estimation , 2005 .

[2]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[3]  Andy Oram,et al.  Peer-to-Peer: Harnessing the Power of Disruptive Technologies , 2001 .

[4]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[5]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[6]  Gustavo de Veciana,et al.  Service capacity of peer to peer networks , 2004, IEEE INFOCOM 2004.

[7]  Helen J. Wang,et al.  Server-based inference of Internet link lossiness , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[8]  S Redner,et al.  Degree distributions of growing networks. , 2001, Physical review letters.

[9]  Catherine Rosenberg,et al.  Analysis of parallel downloading for large file distribution , 2003, The Ninth IEEE Workshop on Future Trends of Distributed Computing Systems, 2003. FTDCS 2003. Proceedings..

[10]  Eli Upfal,et al.  Building low-diameter P2P networks , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[11]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[12]  Mark Crovella,et al.  Virtual landmarks for the internet , 2003, IMC '03.

[13]  Beom Jun Kim,et al.  Path finding strategies in scale-free networks. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Jia Wang,et al.  Analyzing peer-to-peer traffic across large networks , 2004, IEEE/ACM Trans. Netw..

[15]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[16]  Ion Stoica,et al.  Towards a More Functional and Secure Network Infrastructure , 2003 .

[17]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[18]  Krishna P. Gummadi,et al.  King: estimating latency between arbitrary internet end hosts , 2002, IMW '02.

[19]  Scott Shenker,et al.  Can Heterogeneity Make Gnutella Scalable? , 2002, IPTPS.

[20]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[21]  Rayadurgam Srikant,et al.  Modeling and performance analysis of BitTorrent-like peer-to-peer networks , 2004, SIGCOMM 2004.

[22]  Diomidis Spinellis,et al.  A survey of peer-to-peer content distribution technologies , 2004, CSUR.

[23]  Paul Francis,et al.  IDMaps: a global internet host distance estimation service , 2001, TNET.

[24]  Jeffrey Considine,et al.  Informed content delivery across adaptive overlay networks , 2002, IEEE/ACM Transactions on Networking.

[25]  Lada A. Adamic The Small World Web , 1999, ECDL.

[26]  J. Ritter Why Gnutella Can't Scale. No, Really , 2001 .

[27]  H E Stanley,et al.  Classes of small-world networks. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Krishna P. Gummadi,et al.  Measurement, modeling, and analysis of a peer-to-peer file-sharing workload , 2003, SOSP '03.

[29]  Jérôme Lacan,et al.  When FEC Speed up Data Access in P2P Networks , 2002, IDMS/PROMS.

[30]  Ben Y. Zhao,et al.  Maintenance-Free Global Data Storage , 2001, IEEE Internet Comput..

[31]  Rita H. Wouhaybi,et al.  Incentive engineering in wireless LAN based access networks , 2002, 10th IEEE International Conference on Network Protocols, 2002. Proceedings..

[32]  Jeffrey S. Chase,et al.  Correlating Instrumentation Data to System States: A Building Block for Automated Diagnosis and Control , 2004, OSDI.

[33]  Pablo Rodriguez,et al.  Parallel-access for mirror sites in the Internet , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[34]  John Kubiatowicz,et al.  Erasure Coding Vs. Replication: A Quantitative Comparison , 2002, IPTPS.

[35]  Gudmund R. Iversen,et al.  Bayesian statistical inference , 1984 .

[36]  Dan S. Wallach,et al.  Economic Behavior of Peer-to-Peer Storage Networks , 2003 .

[37]  Mostafa Ammar,et al.  On the effect of large-scale deployment of parallel downloading , 2003, Proceedings the Third IEEE Workshop on Internet Applications. WIAPP 2003.

[38]  David G. Andersen,et al.  Proceedings of Usits '03: 4th Usenix Symposium on Internet Technologies and Systems Mayday: Distributed Filtering for Internet Services , 2022 .

[39]  Seif Haridi,et al.  Efficient Broadcast in Structured P2P Networks , 2003, IPTPS.

[40]  Ellen W. Zegura,et al.  Network Measurement as a Cooperative Enterprise , 2002, IPTPS.

[41]  Lada A. Adamic,et al.  Local Search in Unstructured Networks , 2002, ArXiv.

[42]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[43]  Mihajlo A. Jovanović,et al.  Modeling Large-scale Peer-to-Peer Networks and a Case Study of Gnutella , 2001 .

[44]  Emin Gün Sirer,et al.  Meridian: a lightweight network location service without virtual coordinates , 2005, SIGCOMM '05.

[45]  Randy H. Katz,et al.  Dynamic Replica Placement for Scalable Content Delivery , 2002, IPTPS.

[46]  Jon Crowcroft,et al.  Lighthouses for Scalable Distributed Location , 2003, IPTPS.

[47]  Sujata Banerjee,et al.  Distributed Querying of Internet Distance Information , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[48]  J. Lacan,et al.  Speedup of data access using error correcting codes in peer-to-peer networks , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[49]  Krishna P. Gummadi,et al.  Measurement study of peer-to-peer file system sharing , 2002 .

[50]  Mary Baker,et al.  Attrition Defenses for a Peer-to-Peer Digital Preservation System , 2004, USENIX Annual Technical Conference, General Track.

[51]  Angelos D. Keromytis,et al.  SOS: secure overlay services , 2002, SIGCOMM '02.

[52]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[53]  Nicolas Christin,et al.  Content availability, pollution and poisoning in file sharing peer-to-peer networks , 2005, EC '05.

[54]  Scott Shenker,et al.  Making gnutella-like P2P systems scalable , 2003, SIGCOMM '03.

[55]  Mark Handley,et al.  Topologically-aware overlay construction and server selection , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[56]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[57]  Ian Clarke,et al.  Freenet: A Distributed Anonymous Information Storage and Retrieval System , 2000, Workshop on Design Issues in Anonymity and Unobservability.

[58]  Rita H. Wouhaybi,et al.  Phenix: supporting resilient low-diameter peer-to-peer topologies , 2004, IEEE INFOCOM 2004.

[59]  Jérôme Lacan,et al.  Enhancing Peer-to-Peer Parallel Data Access with PeerFecT , 2003, Networked Group Communication.

[60]  Micah Adler,et al.  Optimal Peer Selection in a Free-Market Peer- Resource Economy , 2004 .

[61]  Reinhard Diestel,et al.  Graph Theory , 1997 .

[62]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[63]  Edith Cohen,et al.  Replication strategies in unstructured peer-to-peer networks , 2002, SIGCOMM.

[64]  Marvin Theimer,et al.  Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs , 2000, SIGMETRICS '00.

[65]  Mostafa H. Ammar,et al.  A reputation system for peer-to-peer networks , 2003, NOSSDAV '03.

[66]  Sridhar Srinivasan,et al.  M-coop: a scalable infrastructure for network measurement , 2003, Proceedings the Third IEEE Workshop on Internet Applications. WIAPP 2003.

[67]  Aviel D. Rubin,et al.  Publius: a robust, tamper-evident, censorship-resistant web publishing system , 2000 .

[68]  Hector Garcia-Molina,et al.  The Eigentrust algorithm for reputation management in P2P networks , 2003, WWW '03.

[69]  Shlomo Zilberstein,et al.  Adaptive Peer Selection , 2003, IPTPS.

[70]  G. Hardin,et al.  The Tragedy of the Commons , 1968, Green Planet Blues.

[71]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[72]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[73]  Ling Liu,et al.  Building Trust in Decentralized Peer-to-Peer Electronic Communities , 2002 .

[74]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.