Online detection of network traffic anomalies using behavioral distance

While network-wide anomaly analysis has been well studied, the on-line detection of network traffic anomalies at a vantage point inside the Internet still poses quite a challenge to network administrators. In this paper, we develop a behavioral distance based anomaly detection mechanism with the capability of performing on-line traffic analysis. To construct accurate online traffic profiles, we introduce horizontal and vertical distance metrics between various traffic features (i.e., packet header fields) in the traffic data streams. The significant advantages of the proposed approach lie in four aspects: (1) it is efficient and simple enough to process on-line traffic data; (2) it facilitates protocol behavioral analysis without maintaining per-flow state; (3) it is scalable to high speed traffic links because of the aggregation, and (4) using various combinations of packet features and measuring distances between them, it is capable for accurate on-line anomaly detection. We validate the efficacy of our proposed detection system by using network traffic traces collected at Abilene and MAWI high-speed links.

[1]  Vyas Sekar,et al.  Data streaming algorithms for estimating entropy of network traffic , 2006, SIGMETRICS '06/Performance '06.

[2]  Jake D. Brutlag,et al.  Aberrant Behavior Detection in Time Series for Network Monitoring , 2000, LISA.

[3]  John W. Lockwood,et al.  Deep packet inspection using parallel bloom filters , 2004, IEEE Micro.

[4]  Alex C. Snoeren,et al.  Hash-based IP traceback , 2001, SIGCOMM '01.

[5]  Ramesh Govindan,et al.  Detection and identification of network anomalies using sketch subspaces , 2006, IMC '06.

[6]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[7]  Albert G. Greenberg,et al.  Network anomography , 2005, IMC '05.

[8]  Graham Cormode,et al.  What's new: finding significant differences in network data streams , 2004, IEEE/ACM Transactions on Networking.

[9]  Paul Barford,et al.  A signal analysis of network traffic anomalies , 2002, IMW '02.

[10]  Yan Chen,et al.  Reversible sketches for efficient and accurate change detection over network data streams , 2004, IMC '04.

[11]  Jennifer Rexford,et al.  Sensitivity of PCA for traffic anomaly detection , 2007, SIGMETRICS '07.

[12]  Kavé Salamatian,et al.  Combining filtering and statistical methods for anomaly detection , 2005, IMC '05.

[13]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[14]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[15]  Carsten Lund,et al.  Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications , 2004, IMC '04.

[16]  Kang G. Shin,et al.  Detecting SYN flooding attacks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[17]  Donald F. Towsley,et al.  Detecting anomalies in network traffic using maximum entropy estimation , 2005, IMC '05.

[18]  Carsten Lund,et al.  Properties and prediction of flow statistics from sampled packet streams , 2002, IMW '02.

[19]  George Varghese,et al.  On Scalable Attack Detection in the Network , 2004, IEEE/ACM Transactions on Networking.

[20]  Vern Paxson,et al.  Enhancing byte-level network intrusion detection signatures with context , 2003, CCS '03.

[21]  Zhi-Li Zhang,et al.  A Real-Time Network Traffic Profiling System , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[22]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[23]  Mark Crovella,et al.  Mining anomalies using traffic feature distributions , 2005, SIGCOMM '05.