Network anomaly detection using autonomous system flow aggregates

Detecting malicious traffic streams in modern computer networks is a challenging task due to the growing traffic volume that must be analyzed. Traditional anomaly detection systems based on packet inspection face a scalability problem in terms of computational and storage capacity. One solution to this scalability problem is to analyze traffic based on IP flow aggregates. However, IP aggregates can still result in prohibitively large datasets for networks with heavy traffic loads. In this paper, we investigate whether anomaly detection is still possible when traffic is aggregated at a coarser scale. We propose a volumetric analysis methodology that aggregates traffic at the Autonomous System (AS) level. We show that our methodology reduces the number of flows to be analyzed by several orders of magnitude compared with IP flow level analysis, while still detecting traffic anomalies.

[1]  Mark Crovella,et al.  Characterization of network-wide anomalies in traffic flows , 2004, IMC '04.

[2]  Wanlei Zhou,et al.  Discriminating DDoS Flows from Flash Crowds Using Information Distance , 2009, 2009 Third International Conference on Network and System Security.

[3]  Richard P. Lippmann,et al.  1999 DARPA Intrusion Detection Evaluation: Design and Procedures , 2001 .

[4]  Jing Wang,et al.  Network anomaly detection: A survey and comparative analysis of stochastic and deterministic methods , 2013, 52nd IEEE Conference on Decision and Control.

[5]  Gerhard Münz,et al.  Flexible Flow Aggregation for Adaptive Network Monitoring , 2006, Proceedings. 2006 31st IEEE Conference on Local Computer Networks.

[6]  Enrico Gregori,et al.  Inferring geography from BGP raw data , 2012, 2012 Proceedings IEEE INFOCOM Workshops.

[7]  Donald F. Towsley,et al.  Detecting anomalies in network traffic using maximum entropy estimation , 2005, IMC '05.

[8]  Urbashi Mitra,et al.  Parametric Methods for Anomaly Detection in Aggregate Traffic , 2011, IEEE/ACM Transactions on Networking.

[9]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[10]  Jugal K. Kalita,et al.  An effective unsupervised network anomaly detection method , 2012, ICACCI '12.

[11]  V Jyothsna,et al.  A Review of Anomaly based Intrusion Detection Systems , 2011 .

[12]  Shunji Abe,et al.  Detecting DoS attacks using packet size distribution , 2007, 2007 2nd Bio-Inspired Models of Network, Information and Computing Systems.

[13]  Manish Kumar,et al.  A flow based anomaly detection system using chi-square technique , 2010, 2010 IEEE 2nd International Advance Computing Conference (IACC).

[14]  Wanlei Zhou,et al.  Low-Rate DDoS Attacks Detection and Traceback by Using New Information Metrics , 2011, IEEE Transactions on Information Forensics and Security.

[15]  Vasilios A. Siris,et al.  Application of anomaly detection algorithms for detecting SYN flooding attacks , 2004, GLOBECOM.

[16]  Jugal K. Kalita,et al.  Network Anomaly Detection: Methods, Systems and Tools , 2014, IEEE Communications Surveys & Tutorials.

[17]  Min Sik Kim,et al.  Using TCAM efficiently for IP route lookup , 2011, 2011 IEEE Consumer Communications and Networking Conference (CCNC).

[18]  Steve Uhlig,et al.  Towards TCAM-based scalable virtual routers , 2012, CoNEXT '12.

[19]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.