Interlaced: Fully decentralized churn stabilization for Skip Graph-based DHTs

As a distributed hash table (DHT) routing overlay, Skip Graph is used in a variety of peer-to-peer (P2P) systems including cloud storage, social networks, and search engines. The overlay connectivity of P2P systems is negatively affected by the arrivals and departures of nodes to and from the system that is known as churn. Preserving connectivity of the overlay network (i.e., the reachability of every pair of nodes) under churn is a performance challenge in every P2P system including the Skip Graph-based ones. The existing decentralized churn stabilization solutions that are applicable on Skip Graphs have intensive communication complexities, which leave them unable to provide a strong overlay connectivity, especially under high rates of churn. In this paper, we propose Interlaced, a fully decentralized churn stabilization mechanism for Skip Graphs that provides drastically stronger overlay connectivity without changing the asymptotic complexity of the Skip Graph in terms of storage, computation, and communication. We also propose the Sliding Window De Bruijn Graph (SW-DBG) as a tool to predict the availability of nodes with high accuracy. Our simulation results show that in comparison to the best existing DHT-based solutions, Interlaced improves the overlay connectivity of Skip Graph under churn with the gain of about 1.81 times. A Skip Graph that benefits from Interlaced and SW-DBG is about 2.47 times faster on average in routing the queries under churn compared to the best existing solutions. We also present an adaptive extension of Interlaced to be applied on other DHTs, for example Kademlia.

[1]  Vwani P. Roychowdhury,et al.  Resilience of structured P2P systems under churn: The reachable component method , 2008, Comput. Commun..

[2]  Arno Wacker,et al.  Evaluating Connection Resilience for the Overlay Network Kademlia , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[3]  Alptekin Küpçü,et al.  Locality Aware Skip Graph , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems Workshops.

[4]  Tomoki Yoshihisa,et al.  A P2P Sensor Data Stream Delivery System to Accommodate Heterogeneous Cycles Using Skip Graphs , 2015, 2015 10th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC).

[5]  Shalini Batra,et al.  A short survey of Advantages and Applications of Skip Graphs , 2013 .

[6]  Daniel Stutzbach,et al.  Understanding churn in peer-to-peer networks , 2006, IMC '06.

[7]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[8]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[9]  de Ng Dick Bruijn.  A combinatorial problem , 1946 .

[10]  Elizabeth Pérez Cortés,et al.  A performance comparison of Chord and Kademlia DHTs in high churn scenarios , 2015, Peer-to-Peer Netw. Appl..

[11]  Öznur Özkasap,et al.  Guard: Secure routing in skip graph , 2017, 2017 IFIP Networking Conference (IFIP Networking) and Workshops.

[12]  Jia Wang,et al.  Analyzing peer-to-peer traffic across large networks , 2002, IMW '02.

[13]  Hanan Lutfiyya,et al.  Decentralized approach to resource availability prediction using group availability in a P2P desktop grid , 2012, Future Gener. Comput. Syst..

[14]  Vladimir Vlassov,et al.  Interaction between Network Partitioning and Churn in a Self-Healing Structured Overlay Network , 2015, 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS).

[15]  Yalin Ding,et al.  Using the complementary nature of node joining and leaving to handle churn problem in P2P networks , 2013, Comput. Electr. Eng..

[16]  Edith Cohen,et al.  Search and replication in unstructured peer-to-peer networks , 2002, ICS '02.

[17]  Jianjun Yu,et al.  pService: Towards Similarity Search on Peer-to-Peer Web Services Discovery , 2009, 2009 First International Conference on Advances in P2P Systems.

[18]  John R. Douceur.  Is remote host availability governed by a universal law? , 2003, PERV.

[19]  Alptekin Küpçü,et al.  Decentralized Utility- and Locality-Aware Replication for Heterogeneous DHT-Based P2P Cloud Storage Systems , 2019, IEEE Transactions on Parallel and Distributed Systems.

[20]  Valerio Schiavoni,et al.  Exploiting Node Connection Regularity for DHT Replication , 2011, 2011 IEEE 30th International Symposium on Reliable Distributed Systems.

[21]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[22]  Robert Tappan Morris,et al.  Bandwidth-efficient management of DHT routing tables , 2005, NSDI.

[23]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[24]  Christian Scheideler,et al.  Tiara: A self-stabilizing deterministic skip list and skip graph , 2012, Theor. Comput. Sci..

[25]  Michael T. Goodrich,et al.  The rainbow skip graph: a fault-tolerant constant-degree distributed data structure , 2006, SODA '06.

[26]  Yong Meng Teo,et al.  An adaptive stabilization framework for distributed hash tables , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[27]  Emmanuel Udoh,et al.  Cloud, Grid and High Performance Computing: Emerging Applications , 2011 .

[28]  John N. Tsitsiklis,et al.  Introduction to Probability , 2002 .

[29]  Dmitri Loguinov,et al.  On Lifetime-Based Node Failure and Stochastic Resilience of Decentralized Peer-to-Peer Networks , 2005, IEEE/ACM Transactions on Networking.

[30]  Horst Rinne,et al.  The Weibull Distribution: A Handbook , 2008 .

[31]  Jim Dowling,et al.  Discovery of Stable Peers in a Self-organising Peer-to-Peer Gradient Topology , 2006, DAIS.

[32]  Alptekin Küpçü,et al.  Decentralized and locality aware replication method for DHT-based P2P storage systems , 2018, Future Gener. Comput. Syst..

[33]  Meina Song,et al.  A Cloud Computing Platform Based on P2P , 2009, 2009 IEEE International Symposium on IT in Medicine & Education.

[34]  Feng Xiao,et al.  Exploring peer heterogeneity: Towards understanding and application , 2011, 2011 IEEE International Conference on Peer-to-Peer Computing.

[35]  Taieb Znati,et al.  Modeling Churn in P2P Networks , 2007, 40th Annual Simulation Symposium (ANSS'07).

[36]  Rajmohan Rajaraman,et al.  Accessing Nearby Copies of Replicated Objects in a Distributed Environment , 1997, SPAA '97.

[37]  Joan Manuel Marquès,et al.  Long-term availability prediction for groups of volunteer resources , 2012, J. Parallel Distributed Comput..

[38]  Alptekin Küpçü,et al.  Awake: Decentralized and Availability Aware Replication for P2P Cloud Storage , 2016, 2016 IEEE International Conference on Smart Cloud (SmartCloud).

[39]  S. D. Madhu Kumar,et al.  Authenticated and persistent skip graph: a data structure for cloud based data-centric applications , 2012, ICACCI '12.

[40]  Hideki Tode,et al.  Autonomous and distributed construction of locality aware skip graph , 2017, 2017 14th IEEE Annual Consumer Communications & Networking Conference (CCNC).

[41]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[42]  Krishna P. Gummadi,et al.  Measurement, modeling, and analysis of a peer-to-peer file-sharing workload , 2003, SOSP '03.

[43]  Alptekin Küpçü,et al.  SkipSim: Scalable Skip Graph Simulator , 2020, 2020 IEEE International Conference on Blockchain and Cryptocurrency (ICBC).

[44]  Satoshi Fukumoto,et al.  Fast and secure tag authentication in large-scale RFID systems using skip graphs , 2018, Comput. Commun..

[45]  Alptekin Küpçü,et al.  LightChain: A DHT-based Blockchain for Resource Constrained Environments , 2019, IACR Cryptol. ePrint Arch..

[46]  Alptekin Küpçü,et al.  LARAS: Locality aware replication algorithm for the Skip Graph , 2016, NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium.

[47]  Erik D. Demaine,et al.  EpiChord: parallelizing the chord lookup algorithm with reactive routing state management , 2004, Proceedings. 2004 12th IEEE International Conference on Networks (ICON 2004) (IEEE Cat. No.04EX955).

[48]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[49]  Brian D. Noble,et al.  Exploiting Availability Prediction in Distributed Systems , 2006, NSDI.

[50]  Daeil Seo,et al.  Replica Placement Algorithm for Highly Available Peer-to-Peer Storage Systems , 2009, 2009 First International Conference on Advances in P2P Systems.

[51]  Amrit Lal Sangal,et al.  Performance Analysis of Predictive Stabilization for Churn Handling in Structured Overlay Networks , 2016, BDAW '16.

[52]  Öznur Özkasap,et al.  ProFID: Practical frequent items discovery in peer-to-peer networks , 2013, Future Gener. Comput. Syst..

[53]  Effects of Churn on Structured P 2 P Overlay Networks , 2014 .

[54]  Öznur Özkasap,et al.  ELATS: Energy and locality aware aggregation tree for skip graph , 2017, 2017 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom).

[55]  Scott Shenker,et al.  Minimizing churn in distributed systems , 2006, SIGCOMM.

[56]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[57]  Robert Tappan Morris,et al.  Comparing the Performance of Distributed Hash Tables Under Churn , 2004, IPTPS.

[58]  Seif Haridi,et al.  DKS(N, k, f): a family of low communication, scalable and fault-tolerant infrastructures for P2P applications , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[59]  Michiharu Takemoto,et al.  SFB: a scalable method for handling range queries on Skip Graphs , 2015 .

[60]  Erwan Le Merrer,et al.  Finding Good Partners in Availability-Aware P2P Networks , 2009, SSS.

[61]  Fabián E. Bustamante,et al.  Designing Less-Structured P2P Systems for the Expected High Churn , 2008, IEEE/ACM Transactions on Networking.

[62]  Xiaoning Ding,et al.  Measurements, analysis, and modeling of BitTorrent-like systems , 2005, IMC '05.

[63]  Christian Scheideler,et al.  SKIP+: A Self-Stabilizing Skip Graph , 2014, JACM.

[64]  Brian D. Noble,et al.  Improving distributed system performance using machine availability prediction , 2006, PERV.

[65]  Fabián E. Bustamante,et al.  Friendships that Last: Peer Lifespan and its Role in P2P Protocols , 2003, WCW.

[66]  Zhanhuai Li,et al.  Study on Cloud Storage System Based on Distributed Storage Systems , 2010, 2010 International Conference on Computational and Information Sciences.

[67]  Michael B. Jones,et al.  SkipNet: A Scalable Overlay Network with Practical Locality Properties , 2003, USENIX Symposium on Internet Technologies and Systems.

[68]  Juan Julián Merelo Guervós,et al.  Resilience to churn of a peer-to-peer evolutionary algorithm , 2008, Int. J. High Perform. Syst. Archit..

[69]  A Persistent Structured Hierarchical Overlay Network to Counter Intentional Churn Attack , 2016, J. Comput. Networks Commun..

[70]  Eli Upfal,et al.  Building low-diameter peer-to-peer networks , 2003, IEEE J. Sel. Areas Commun..

[71]  Karl Aberer,et al.  Internet-Scale Storage Systems under Churn -- A Study of the Steady-State using Markov Models , 2006, Sixth IEEE International Conference on Peer-to-Peer Computing (P2P'06).

[72]  Kam-Wing Ng,et al.  Stochastic analysis of the interplay between object maintenance and churn , 2008, Comput. Commun..

[73]  Cláudio L. Amorim,et al.  D1HT: a distributed one hop hash table , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[74]  A. L. Sangal,et al.  Modeling and simulation of adaptive Neuro-fuzzy based intelligent system for predictive stabilization in structured overlay networks , 2017 .

[75]  Kazuyuki Shudo,et al.  FRT-Skip Graph: A Skip Graph-style structured overlay based on Flexible Routing Tables , 2016, 2016 IEEE Symposium on Computers and Communication (ISCC).

[76]  Kai Hwang,et al.  Cloud Security with Virtualized Defense and Reputation-Based Trust Mangement , 2009, 2009 Eighth IEEE International Conference on Dependable, Autonomic and Secure Computing.

[77]  Ramanpreet Kaur,et al.  Churn handling strategies for structured overlay networks: A survey , 2017, Multiagent Grid Syst..

[78]  Mika Ylianttila,et al.  Effects of different churn models on the performance of structured peer-to-peer networks , 2009, 2009 IEEE 20th International Symposium on Personal, Indoor and Mobile Radio Communications.

[79]  Raúl Jiménez,et al.  Connectivity properties of Mainline BitTorrent DHT nodes , 2009, 2009 IEEE Ninth International Conference on Peer-to-Peer Computing.

[80]  Sébastien Tixeuil,et al.  Infinite Unlimited Churn , 2016, ArXiv.