Anomaly detection in IP networks based on randomized subspace methods

In this paper we propose novel randomized subspace methods to detect anomalies in Internet Protocol networks. Given a data matrix containing information about network traffic, the proposed approaches perform a normal-plus-anomalous matrix decomposition aided by the randomized sampling scheme and subsequently detect traffic anomalies in the anomalous subspace using a statistical test. Simulation results demonstrate improvement over the traditional principal component analysis-based subspace methods in terms of robustness to noise and detection rate.

[1]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[2]  J. Edward Jackson,et al.  A User's Guide to Principal Components. , 1991 .

[3]  Rodrigo C. de Lamare,et al.  Switched-randomized robust PCA for foreground and background separation in video surveillance , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[4]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[5]  Walter Willinger,et al.  Spatio-Temporal Compressive Sensing and Internet Traffic Matrices (Extended Version) , 2012, IEEE/ACM Transactions on Networking.

[6]  Marina Thottan,et al.  Anomaly detection in IP networks , 2003, IEEE Trans. Signal Process..

[7]  Per-Gunnar Martinsson,et al.  On the Compression of Low Rank Matrices , 2005, SIAM J. Sci. Comput..

[8]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[9]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[10]  Y. Vardi,et al.  Network Tomography: Estimating Source-Destination Traffic Intensities from Link Data , 1996 .

[11]  Dacheng Tao,et al.  GoDec: Randomized Lowrank & Sparse Matrix Decomposition in Noisy Case , 2011, ICML.

[12]  Lee D. Davisson,et al.  An Introduction To Statistical Signal Processing , 2004 .

[13]  Mark Tygert,et al.  A Randomized Algorithm for Principal Component Analysis , 2008, SIAM J. Matrix Anal. Appl..

[14]  DiotChristophe,et al.  Diagnosing network-wide traffic anomalies , 2004 .

[15]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[16]  James Demmel,et al.  Fast linear algebra is stable , 2006, Numerische Mathematik.

[17]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[18]  Ling Huang,et al.  In-Network PCA and Anomaly Detection , 2006, NIPS.

[19]  J. E. Jackson,et al.  Control Procedures for Residuals Associated With Principal Component Analysis , 1979 .

[20]  Morteza Mardani,et al.  Estimating Traffic and Anomaly Maps via Network Tomography , 2014, IEEE/ACM Transactions on Networking.

[21]  Jennifer Rexford,et al.  Sensitivity of PCA for traffic anomaly detection , 2007, SIGMETRICS '07.

[22]  Martin May,et al.  Applying PCA for Traffic Anomaly Detection: Problems and Solutions , 2009, IEEE INFOCOM 2009.

[23]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[24]  Albert G. Greenberg,et al.  Network anomography , 2005, IMC '05.

[25]  R. D. Lamare,et al.  Adaptive Reduced-Rank Processing Based on Joint and Iterative Interpolation, Decimation, and Filtering , 2009, IEEE Transactions on Signal Processing.

[26]  S. Joe Qin,et al.  Subspace approach to multidimensional fault identification and reconstruction , 1998 .

[27]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[28]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.