Large-scale monitoring of DHT traffic

Studying deployed Distributed Hash Tables (DHTs) entails monitoring DHT traffic. Commonly, DHT traffic is measured by instrumenting ordinary peers to passively record traffic. In this approach, using a small number of peers leads to a limited (and potentially biased) view of traffic. Alternatively, inserting a large number of peers may disrupt the natural traffic patterns of the DHT and lead to incorrect results. In general, accurately capturing DHT traffic is a challenging task. In this paper, we propose the idea of minimally visible monitors to capture the traffic at a large number of peers with minimum disruption to the DHT. We implement and validate our proposed technique, called Montra, on the Kad DHT. We show that Montra accurately captures around 90% of the query traffic while monitoring roughly 32,000 peers and can accurately identify destination peers for 90% of captured destination traffic. Using Montra, we characterize the traffic in Kad and present our preliminary results.

[1]  Fabián E. Bustamante,et al.  Structured and Unstructured Overlays under the Microscope: A Measurement-based View of Two P2P Systems That People Use , 2006, USENIX Annual Technical Conference, General Track.

[2]  Daniel Stutzbach,et al.  Characterizing files in the modern Gnutella network , 2007, Multimedia Systems.

[3]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[4]  Taoufik En-Najjary,et al.  A global view of kad , 2007, IMC '07.

[5]  Daniel Stutzbach,et al.  Capturing Accurate Snapshots of the Gnutella Network , 2005, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[6]  Robert Tappan Morris,et al.  A performance vs. cost framework for evaluating DHT design tradeoffs under churn , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[7]  Yuval Shavitt,et al.  Geographical Statistics and Characteristics of P2P Query Strings , 2007, IPTPS.

[8]  Krishna P. Gummadi,et al.  The impact of DHT routing geometry on resilience and proximity , 2003, SIGCOMM '03.

[9]  Ghulam Memon gmemon Characterizing Traffic in Widely-Deployed DHT , 2008 .

[10]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[11]  Daniel Stutzbach,et al.  Improving Lookup Performance Over a Widely-Deployed DHT , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[12]  Moritz Steiner Load reduction in the KAD peer-to-peer system , 2007 .

[13]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[14]  Thomas E. Anderson,et al.  Profiling a million user dht , 2007, IMC '07.