Multi-low-rank Approximation for Traffic Matrices

With the Internet applications become more complex and diverse, simple network traffic matrix estimation or approximation methods such as gravity model are no longer adequate. In this paper, we advocate a novel approach of approximating traffic matrices with multiple low-rank matrices. We build the theory behind the MULTI-LOW-RANK approximation and discuss the conditions under which it is better than Low-Rank SVD in terms of both matrix approximation and preserving the local (and global) structure of the traffic matrices. Further, we develop an effective technique based on spectral clustering of column/row feature vectors for decomposing traffic matrices into multiple low rank matrices. We perform a series of experiments on traffic matrices extracted from a synthetic dataset and two real world datasets – one that represents nationwide cellular traffic and another taken from a tier-1 ISP. The results thus obtained show that: 1) MULTI-LOW-RANK approximation is superior for traffic classification; 2) it can be used to predict complete or missing entries of traffic matrices over time; 3) show it's robustness against noise; and 4) demonstrate that it closely follows the optimal solution (i.e., low-rank SVD solution).

[1]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[2]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[3]  Shaojie Tang,et al.  Time series matrix factorization prediction of internet traffic matrices , 2012, 37th Annual IEEE Conference on Local Computer Networks.

[4]  Albert G. Greenberg,et al.  Network anomography , 2005, IMC '05.

[5]  Vijay Erramilli,et al.  An independent-connection model for traffic matrices , 2006, IMC '06.

[6]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[7]  Zhi-Li Zhang,et al.  From traffic matrix to routing matrix: PoP level traffic characteristics for a Tier-1 ISP , 2011, PERV.

[8]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[9]  Matthew Roughan,et al.  Computation of IP traffic from link , 2003, SIGMETRICS 2003.

[10]  Kavé Salamatian,et al.  Traffic matrix estimation: existing techniques and new directions , 2002, SIGCOMM '02.

[11]  Baolin Yin,et al.  Structural analysis of network traffic matrix via relaxed principal component pursuit , 2011, Comput. Networks.

[12]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[13]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[14]  Konstantina Papagiannaki,et al.  Traffic matrices: balancing measurements, inference and modeling , 2005, SIGMETRICS '05.

[15]  Albert G. Greenberg,et al.  Fast accurate computation of large-scale IP traffic matrices from link loads , 2003, SIGMETRICS '03.

[16]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[17]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[18]  WillingerWalter,et al.  Spatio-temporal compressive sensing and internet traffic matrices , 2009 .

[19]  Walter Willinger,et al.  Spatio-temporal compressive sensing and internet traffic matrices , 2009, SIGCOMM '09.

[20]  C. Berenstein,et al.  Network tomography , 2007 .

[21]  Fabien Moutarde,et al.  Analysis of network-level traffic states using locality preservative non-negative matrix factorization , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[22]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[23]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.