Internet Traffic Matrix Structural Analysis Based on Multi-Resolution RPCA

The Internet traffic matrix plays a significant roll in network operation and management, therefore, the structural analysis of traffic matrix, which decomposes different traffic components of this high-dimensional traffic dataset, is quite valuable to some network applications. In this study, based on the Robust Principal Component Analysis (RPCA) theory, a novel traffic matrix structural analysis approach named Multi-Resolution RPCA is created, which utilizes the wavelet multi-resolution analysis. Firstly, we build the Multi-Resolution Traffic Matrix Decomposition Model (MR-TMDM), which characterizes the smoothness of the deterministic traffic by its wavelet coefficients. Secondly, based on this model, we improve the Stable Principal Component Pursuit (SPCP), propose a new traffic matrix decomposition method named SPCP-MRC with Multi-Resolution Constraints, and design its numerical algorithm. Specifically, we give and prove the closed-form solution to a sub-problem in the algorithm. Lastly, we evaluate different traffic decomposition methods by multiple groups of simulated traffic matrices containing different kinds of anomalies and distinct noise levels. It is demonstrated that SPCP-MRC, compared with other methods, achieves more accurate and more reasonable traffic decompositions.

[1]  Jennifer Rexford,et al.  Sensitivity of PCA for traffic anomaly detection , 2007, SIGMETRICS '07.

[2]  Baolin Yin,et al.  Structural analysis of network traffic matrix via relaxed principal component pursuit , 2011, Comput. Networks.

[3]  Patrice Abry,et al.  A Wavelet-Based Joint Estimator of the Parameters of Long-Range Dependence , 1999, IEEE Trans. Inf. Theory.

[4]  Anura P. Jayasumana,et al.  Extracting baseline patterns in Internet traffic using Robust Principal Components , 2011, 2011 IEEE 36th Conference on Local Computer Networks.

[5]  Y. Peng De-noising by modified soft-thresholding , 2000, IEEE APCCAS 2000. 2000 IEEE Asia-Pacific Conference on Circuits and Systems. Electronic Communication Systems. (Cat. No.00EX394).

[6]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[7]  Baolin Yin,et al.  An Improved Traffic Matrix Decomposition Method with Frequency Domain Regularization , 2013, IEICE Trans. Inf. Syst..

[8]  Wei Wang,et al.  Robust traffic anomaly detection with principal component pursuit , 2010, CoNEXT '10 Student Workshop.

[9]  A.H. Tewfik,et al.  Correlation structure of the discrete wavelet coefficients of fractional Brownian motion , 1992, IEEE Trans. Inf. Theory.

[10]  Steve Uhlig,et al.  Providing public intradomain traffic matrices to the research community , 2006, CCRV.

[11]  Emilio Leonardi,et al.  Estimating dynamic traffic matrices by using viable routing changes , 2007 .

[12]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[13]  Ling Huang,et al.  ANTIDOTE: understanding and defending against poisoning of anomaly detectors , 2009, IMC '09.

[14]  Paris Smaragdis,et al.  Singing-voice separation from monaural recordings using robust principal component analysis , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Yi Ma,et al.  TILT: Transform Invariant Low-Rank Textures , 2010, ACCV.

[16]  Paul Barford,et al.  A signal analysis of network traffic anomalies , 2002, IMW '02.

[17]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[18]  Kavé Salamatian,et al.  Combining filtering and statistical methods for anomaly detection , 2005, IMC '05.

[19]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[20]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[21]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[22]  Matthew Roughan,et al.  Simplifying the synthesis of internet traffic matrices , 2005, CCRV.

[23]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[24]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[25]  John Wright,et al.  Decomposing background topics from keywords by principal component pursuit , 2010, CIKM.

[26]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[27]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[28]  Matthew Roughan,et al.  The need for simulation in evaluating anomaly detectors , 2008, CCRV.

[29]  Steve Uhlig,et al.  On the complexity of Internet traffic dynamics on its topology , 2010, Telecommun. Syst..

[30]  Arvind Ganesh,et al.  Fast algorithms for recovering a corrupted low-rank matrix , 2009, 2009 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[31]  Mark Crovella,et al.  Characterization of network-wide anomalies in traffic flows , 2004, IMC '04.

[32]  John Wright,et al.  RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Xiangliang Zhang,et al.  Fast intrusion detection based on a non-negative matrix factorization model , 2009, J. Netw. Comput. Appl..

[34]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.