Pinpoint : Identifying Packet Loss Culprits Using Adaptive Sampling

Accurately estimating all link-level properties of a large network has proven to be very difficult. The measurements used for these estimates require significant collaboration from all endpoints on the network, significantly reducing their applicability for large scale Internet measurements. We present a scalable approach using a small number of hosts without collaboration from existing routers and minimal collaboration between the hosts. Our approach is based on adaptive sampling. Initially, each host probes a set of receivers at a low frequency. When packet losses are detected, the sampling rate increases. By detecting correlations between time series and combining them with information about network connectivity, the host identifies a set of suspected lossy routers. Hosts then communicate with each other, combining evidence to identify routers with high packet loss. Our experiments show that using a relatively small set of hosts and receivers, we can gather sufficient evidence to identify a small number of routers that cause most of the packet loss in a geographically diverse sample of the Internet. We deployed our method for one month on 68 PlanetLab nodes. As a result of that deployment, we identified 128 routers of the ≈ 4,500 accounting for 87% of the ob-

[1]  Donald F. Towsley,et al.  Multicast-based loss inference with missing data , 2002, IEEE J. Sel. Areas Commun..

[2]  Robert Nowak,et al.  Internet tomography , 2002, IEEE Signal Process. Mag..

[3]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[4]  Alejandro López-Ortiz,et al.  On the number of distributed measurement points for network tomography , 2003, IMC '03.

[5]  Prasad Calyam,et al.  Performance Measurement and Analysis of H.323 Traffic , 2004, PAM.

[6]  Ratul Mahajan,et al.  User-level internet path diagnosis , 2003, SOSP '03.

[7]  Donald F. Towsley,et al.  Multicast-based inference of network-internal loss characteristics , 1999, IEEE Trans. Inf. Theory.

[8]  Sally Floyd,et al.  Why we don't know how to simulate the Internet , 1997, WSC '97.

[9]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[10]  Chun-Ying Huang,et al.  Quantifying Skype user satisfaction , 2006, SIGCOMM.

[11]  Vern Paxson,et al.  End-to-end routing behavior in the Internet , 1996, TNET.

[12]  Paul Barford,et al.  Improving accuracy in end-to-end packet loss measurement , 2005, SIGCOMM '05.

[13]  Yong-June Shin,et al.  A wavelet-based approach to detect shared congestion , 2004, TNET.

[14]  Donald F. Towsley,et al.  Network tomography on general topologies , 2002, SIGMETRICS '02.

[15]  Yao Zhao,et al.  Towards unbiased end-to-end network diagnosis , 2009, TNET.

[16]  Don Towsley,et al.  The use of end-to-end multicast measurements for characterizing internal network behavior , 2000, IEEE Commun. Mag..

[17]  François Baccelli,et al.  The Role of PASTA in Network Measurement , 2006, IEEE/ACM Transactions on Networking.

[18]  Robert Nowak,et al.  Network Tomography: Recent Developments , 2004 .

[19]  Stuart Barber,et al.  All of Statistics: a Concise Course in Statistical Inference , 2005 .

[20]  Donald F. Towsley,et al.  Detecting shared congestion of flows via end-to-end measurement , 2002, TNET.

[21]  Amin Vahdat,et al.  Detour: informed Internet routing and transport , 1999, IEEE Micro.

[22]  Ronald W. Wolff,et al.  Poisson Arrivals See Time Averages , 1982, Oper. Res..

[23]  Yin Zhang,et al.  On the constancy of internet path properties , 2001, IMW '01.

[24]  Robert Tappan Morris,et al.  The case for resilient overlay networks , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[25]  Helen J. Wang,et al.  Server-based inference of Internet link lossiness , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[26]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[27]  Donald F. Towsley,et al.  Inferring link loss using striped unicast probes , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[28]  Arun Venkataramani,et al.  iPlane: an information plane for distributed services , 2006, OSDI '06.

[29]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[30]  K. Claffy,et al.  Topology discovery by active probing , 2002, Proceedings 2002 Symposium on Applications and the Internet (SAINT) Workshops.

[31]  Vern Paxson,et al.  End-to-end Internet packet dynamics , 1997, SIGCOMM '97.