An Effective Method of Monitoring the Large-Scale Traffic Pattern Based on RMT and PCA

Mechanisms to extract the characteristics of network traffic play a significant role in traffic monitoring, offering helpful information for network management and control. In this paper, a method based on Random Matrix Theory (RMT) and Principal Components Analysis (PCA) is proposed for monitoring and analyzing large-scale traffic patterns in the Internet. Besides the analysis of the largest eigenvalue in RMT, useful information is also extracted from small eigenvalues by a method based on PCA. And then an appropriate approach is put forward to select some observation points on the base of the eigen analysis. Finally, some experiments about peer-to-peer traffic pattern recognition and backbone aggregate flow estimation are constructed. The simulation results show that using about 10% of nodes as observation points, our method can monitor and extract key information about Internet traffic patterns.

[1]  J. Orestes Cerdeira,et al.  Computational aspects of algorithms for variable selection in the context of principal components , 2004, Comput. Stat. Data Anal..

[2]  Fang Hao,et al.  Fast, Memory Efficient Flow Rate Estimation Using Runs , 2007, IEEE/ACM Transactions on Networking.

[3]  E. Wigner,et al.  On the statistical distribution of the widths and spacings of nuclear resonance levels , 1951, Mathematical Proceedings of the Cambridge Philosophical Society.

[4]  Hong Yan,et al.  A clean slate 4D approach to network control and management , 2005, CCRV.

[5]  Suresh Singh,et al.  PAMAS—power aware multi-access protocol with signalling for ad hoc networks , 1998, CCRV.

[6]  Zhao Guo.  Traffic Measurement for Traffic Engineering in IP/MPLS Based Telecom Core Networks , 2002 .

[7]  Anja Feldmann,et al.  The changing nature of network traffic: scaling phenomena , 1998, CCRV.

[8]  Anja Feldmann,et al.  Dynamics of IP traffic: a study of the role of variability and the impact of control , 1999, SIGCOMM '99.

[9]  Michele C. Weigle,et al.  Quantifying the effects of recent protocol improvements to standards-track TCP , 2003, 11th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer Telecommunications Systems, 2003. MASCOTS 2003..

[10]  Shoichiro Asano,et al.  A Study on Detecting Network Anomalies Using Sampled Flow Statistics , 2007, 2007 International Symposium on Applications and the Internet Workshops.

[11]  Jia Liu,et al.  Monitoring the spatial-temporal effect of internet traffic based on Random Matrix Theory , 2008, 2008 33rd IEEE Conference on Local Computer Networks (LCN).

[12]  F. Dyson.  Statistical Theory of the Energy Levels of Complex Systems. I , 1962 .

[13]  Jian Yuan,et al.  Monitoring the macroscopic effect of DDoS flooding attacks , 2005, IEEE Transactions on Dependable and Secure Computing.

[14]  Anirvan M. Sengupta,et al.  Distributions of singular values for some random matrices. , 1997, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[15]  Bernard Gondran,et al.  Large scale cross-correlations in Internet traffic. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Mark Claypool,et al.  Inferring Queue Sizes in Access Networks by Active Measurement , 2004, PAM.

[17]  T. V. Lakshman,et al.  Fast, memory efficient flow rate estimation using runs , 2007, TNET.

[18]  J. Bouchaud,et al.  Noise Dressing of Financial Correlation Matrices , 1998, cond-mat/9810255.

[19]  V. Plerou,et al.  Universal and Nonuniversal Properties of Cross Correlations in Financial Time Series , 1999, cond-mat/9902283.