Deep IP flow inspection to detect beyond network anomalies

Abstract Taking into account the accelerated rate of network growth, the occurrence of anomalies becomes inevitable. A single anomaly can affect the network performance so it is crucial to detect its origin. However, when different kinds of anomalies are present at the same time, it becomes more complicated to detect their root causes. In addition, the network administrator has to deal with questions related to network health, such as bandwidth bottlenecks, and network misuse. Detecting these problems quickly is essential to take appropriate countermeasures. Although many solutions have been proposed to detect anomalies, they do not address other important questions related to network health. In this paper, a system capable of detecting and classifying the anomalies, and extracting detailed information from the network usage, is presented. A graph representation is used, allowing a deep inspection of the IP flows exchanged between the active devices in the network. The Tsallis entropy is applied to detect anomalies. Furthermore, the proposed system allows the network administrator to create metrics to monitor and acquire detailed information about the network equipment, services, and users. Tests using real and artificial datasets demonstrate the effectiveness of the proposed system to detect simultaneous anomalies, and to provide useful information for network-management tasks.

[1]  Marcos V. O. de Assis,et al.  Scorpius: sFlow Network Anomaly Simulator , 2015, J. Comput. Sci..

[2]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[3]  Vladimir Popovskyy,et al.  Entropy methods for DDoS attacks detection in telecommunication systems , 2014, 2014 First International Scientific-Practical Conference Problems of Infocommunications Science and Technology.

[4]  Yonghong Chen,et al.  DDoS Detection Method Based on Chaos Analysis of Network Traffic Entropy , 2014, IEEE Communications Letters.

[5]  Dan Schnackenberg,et al.  Statistical approaches to DDoS attack detection and response , 2003, Proceedings DARPA Information Survivability Conference and Exposition.

[6]  Zubair A. Baig,et al.  An Entropy and Volume-Based Approach for Identifying Malicious Activities in Honeynet Traffic , 2011, 2011 International Conference on Cyberworlds.

[7]  N. Elz,et al.  Anomaly detection using new MIB traffic parameters based on profile , 2012, 2012 8th International Conference on Computing Technology and Information Management (NCM and ICNIT).

[8]  R. Vaarandi.  Detecting anomalous network traffic in organizational private networks , 2013, 2013 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA).

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Vyas Sekar,et al.  An empirical evaluation of entropy-based traffic anomaly detection , 2008, IMC '08.

[11]  S. Selvakumar,et al.  Detection of distributed denial of service attacks using an ensemble of adaptive and hybrid neuro-fuzzy systems , 2013, Comput. Commun..

[12]  Bernhard Plattner,et al.  Entropy based worm and anomaly detection in fast IP networks , 2005, 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprise (WETICE'05).

[13]  Aiko Pras,et al.  An Overview of IP Flow-Based Intrusion Detection , 2010, IEEE Communications Surveys & Tutorials.

[14]  Yang Yahui.  Impact data-exchange based on XML , 2012, 2012 7th International Conference on Computer Science & Education (ICCSE).

[15]  Manish Kumar,et al.  A flow based anomaly detection system using chi-square technique , 2010, 2010 IEEE 2nd International Advance Computing Conference (IACC).

[16]  Xenofontas A. Dimitropoulos,et al.  Histogram-based traffic anomaly detection , 2009, IEEE Transactions on Network and Service Management.

[17]  Pedro Casas,et al.  Volume Anomaly Detection in Data Networks: An Optimal Detection Algorithm vs. the PCA Approach , 2009, FITraMEn.

[18]  Daniel J. Ryan,et al.  Quantifying information security risks using expert judgment elicitation , 2012, Comput. Oper. Res..

[19]  Jiannong Cao,et al.  A TCAM-based solution for integrated traffic anomaly detection and policy filtering , 2009, Comput. Commun..

[20]  Sheng-Hsun Hsu,et al.  Application of SVM and ANN for intrusion detection , 2005, Comput. Oper. Res..

[21]  Artur Ziviani,et al.  Network anomaly detection using nonextensive entropy , 2007, IEEE Communications Letters.

[22]  Sylvio Barbon Junior,et al.  Anomaly detection using digital signature of network segment with adaptive ARIMA model and Paraconsistent Logic , 2014, 2014 IEEE Symposium on Computers and Communications (ISCC).

[23]  James Won-Ki Hong,et al.  A flow-based method for abnormal network traffic detection , 2004, 2004 IEEE/IFIP Network Operations and Management Symposium (IEEE Cat. No.04CH37507).

[24]  E. S. Phalguna Krishna,et al.  An Efficient Flash Crowd Attack Detection to Internet Threat Monitors (ITM) Using Honeypots , 2012, ACITY.

[25]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[26]  Jugal K. Kalita,et al.  Network Anomaly Detection: Methods, Systems and Tools , 2014, IEEE Communications Surveys & Tutorials.

[27]  Jugal K. Kalita,et al.  Surveying Port Scans and Their Detection Methodologies , 2011, Comput. J..

[28]  Muttukrishnan Rajarajan,et al.  Entropy clustering approach for improving forecasting in DDoS attacks , 2015, 2015 IEEE 12th International Conference on Networking, Sensing and Control.

[29]  Eitan Altman,et al.  A survey on networking games in telecommunications , 2006, Comput. Oper. Res..

[30]  Naren Ramakrishnan,et al.  Detection of stealthy malware activities with traffic causality and scalable triggering relation discovery , 2014, AsiaCCS.

[31]  Joel J. P. C. Rodrigues,et al.  A seven-dimensional flow analysis to help autonomous network management , 2014, Inf. Sci..

[32]  Altyeb Altaher,et al.  Real time network anomaly detection using relative entropy , 2011, 8th International Conference on High-capacity Optical Networks and Emerging Technologies.

[33]  Jing Ma,et al.  Network backbone anomaly detection using double random forests based on non-extensive entropy feature extraction , 2013, 2013 Ninth International Conference on Natural Computation (ICNC).

[34]  Martin May,et al.  Impact of packet sampling on anomaly detection metrics , 2006, IMC '06.

[35]  Caiming Liu,et al.  Artificial immunity-based anomaly detection of network user behavior , 2013, 2013 Ninth International Conference on Natural Computation (ICNC).

[36]  David Schneider,et al.  The state of network security , 2012, Netw. Secur..

[37]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[38]  Philippe Owezarski,et al.  Unsupervised Network Intrusion Detection Systems: Detecting the Unknown without Knowledge , 2012, Comput. Commun..

[39]  Weidong Wu,et al.  Online Detection of Network Traffic Anomalies Using Degree Distributions , 2010, Int. J. Commun. Netw. Syst. Sci..

[40]  Kensuke Fukuda,et al.  ADMIRE: Anomaly detection method using entropy-based PCA with three-step sketches , 2013, Comput. Commun..

[41]  Celia Anteneodo,et al.  Nonextensive statistical mechanics and economics , 2003, ArXiv.

[42]  Symeon Papavassiliou,et al.  Network anomaly detection and classification via opportunistic sampling , 2009, IEEE Network.

[43]  S. Mercy Shalinie,et al.  Anomaly detection system based on analysis of packet header and payload histograms , 2011, 2011 International Conference on Recent Trends in Information Technology (ICRTIT).

[44]  Didier Sornette,et al.  Beyond Shannon: Characterizing Internet Traffic with Generalized Entropy Metrics , 2009, PAM.

[45]  Robin Berthier,et al.  Nfsight: netflow-based network awareness tool , 2010 .

[46]  M. Zhanikeev,et al.  Anomaly Identification Based on Flow Analysis , 2006, TENCON 2006 - 2006 IEEE Region 10 Conference.

[47]  Joel J. P. C. Rodrigues,et al.  Autonomous profile-based anomaly detection system using principal component analysis and flow analysis , 2015, Appl. Soft Comput..

[48]  Soung Hie Kim,et al.  Use of multi-attribute decision analysis for designing operations system framework in telecommunications management network , 2000, Comput. Oper. Res..

[49]  Ram Rajagopal,et al.  Characterizing Per-Application Network Traffic Using Entropy , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[50]  Kavé Salamatian,et al.  Anomaly extraction in backbone networks using association rules , 2012, TNET.