Algebraic Approaches for Scalable End-to-End Monitoring and Diagnosis

The rigidity of the Internet architecture led to flourish in the research of end-to-end based systems. In this chapter, we describe a linear algebra-based end-to-end monitoring and diagnosis system. We first propose a tomography-based overlay monitoring system (TOM). Given n end hosts, TOM selectively monitors a basis set of O(nlogn) paths out of all n(n − 1) end-to-end paths. Any end-to-end path can be written as a unique linear combination of paths in the basis set. Consequently, by monitoring loss rates for the paths in the basis set, TOM infers loss rates for all end-to-end paths. Furthermore, leveraging on the scalable measurements from the TOM system, we propose the Least-biased End-to-End Network Diagnosis (in short, LEND) system. We define a minimal identifiable link sequence (MILS) as a link sequence of minimal length whose properties can be uniquely identified from end-to-end measurements. LEND applies an algebraic approach to find out the MILSes and infers the properties of the MILSes efficiently. This also means LEND system achieves the finest diagnosis granularity under the least biased statistical assumptions.

[1]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[2]  Donald F. Towsley,et al.  Multicast-based loss inference with missing data , 2002, IEEE J. Sel. Areas Commun..

[3]  Donald F. Towsley,et al.  Inferring link loss using striped unicast probes , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[4]  Yin Zhang,et al.  On the constancy of internet path properties , 2001, IMW '01.

[5]  Shmuel Friedland,et al.  The Sparse Basis Problem and Multilinear Algebra , 1995, SIAM J. Matrix Anal. Appl..

[6]  Yan Chen,et al.  Towards unbiased end-to-end network diagnosis , 2006, SIGCOMM 2006.

[7]  Mark Handley,et al.  Topologically-aware overlay construction and server selection , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[8]  Randy H. Katz,et al.  On the stability of network distance estimation , 2002, PERV.

[9]  Randy H. Katz,et al.  An algebraic approach to practical and scalable overlay network monitoring , 2004, SIGCOMM 2004.

[10]  Ramesh Govindan,et al.  Heuristics for Internet map discovery , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[11]  Robert Nowak,et al.  Internet tomography , 2002, IEEE Signal Process. Mag..

[12]  Donald F. Towsley,et al.  Network tomography on general topologies , 2002, SIGMETRICS '02.

[13]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[14]  Gene H. Golub,et al.  Matrix computations , 1983 .

[15]  Nick G. Duffield,et al.  Simple network performance tomography , 2003, IMC '03.

[16]  Donald F. Towsley,et al.  Multicast-based inference of network-internal loss characteristics , 1999, IEEE Trans. Inf. Theory.

[17]  Avishai Wool,et al.  Computing the unmeasured: an algebraic approach to Internet mapping , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[18]  Ibrahim Matta,et al.  On the origin of power laws in Internet topologies , 2000, CCRV.

[19]  Yin Zhang,et al.  NetQuest: A Flexible Framework for Large-Scale Network Measurement , 2009, IEEE/ACM Transactions on Networking.

[20]  Mark Crovella,et al.  Efficient monitoring of end-to-end network properties , 2005, INFOCOM.

[21]  Russell R. Barton,et al.  Managing End-to-End Network Performance via Optimized Monitoring Strategies , 2004, Journal of Network and Systems Management.

[22]  Donald F. Towsley,et al.  Multicast-based inference of network-internal characteristics: accuracy of packet loss estimation , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[23]  James Demmel,et al.  LAPACK Users' Guide, Third Edition , 1999, Software, Environments and Tools.

[24]  Yao Zhao,et al.  A Suite of Schemes for User-Level Network Diagnosis without Infrastructure , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[25]  Jack Dongarra,et al.  LAPACK Users' guide (third ed.) , 1999 .

[26]  Helen J. Wang,et al.  Server-based inference of Internet link lossiness , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[27]  Philip K. McKinley,et al.  On the cost-quality tradeoff in topology-aware overlay path probing , 2003, 11th IEEE International Conference on Network Protocols, 2003. Proceedings..

[28]  Ratul Mahajan,et al.  User-level internet path diagnosis , 2003, SOSP '03.

[29]  Don Towsley,et al.  The use of end-to-end multicast measurements for characterizing internal network behavior , 2000, IEEE Commun. Mag..

[30]  Randy H. Katz,et al.  Tomography-based overlay network monitoring , 2003, IMC '03.

[31]  Van Jacobson,et al.  Random early detection gateways for congestion avoidance , 1993, TNET.

[32]  Kostas G. Anagnostakis,et al.  cing: measuring network-internal delays using only existing infrastructure , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[33]  Paul Francis,et al.  IDMaps: a global internet host distance estimation service , 2001, TNET.