An Abnormal Network Traffic Detection Algorithm Based on Big Data Analysis

Anomaly network detection is a very important way to analyze and detect malicious behavior in network. How to effectively detect anomaly network flow under the pressure of big data is a very important area, which has attracted more and more researchers’ attention. In this paper, we propose a new model based on big data analysis, which can avoid the influence brought by adjustment of network traffic distribution, increase detection accuracy and reduce the false negative rate. Simulation results reveal that, compared with k-means, decision tree and random forest algorithms, the proposed model has a much better performance, which can achieve a detection rate of 95.4% on normal data, 98.6% on DoS attack, 93.9% on Probe attack, 56.1% on U2R attack, and 77.2% on R2L attack.

[1]  R.K. Cunningham,et al.  Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[2]  Stefan Axelsson Research in Intrusion-Detection Systems: A Survey , 1998 .

[3]  Wei Fan,et al.  Mining big data: current status, and forecast to the future , 2013, SKDD.

[4]  Shankar Chakraborty,et al.  A knowledge-based system for end mill selection , 2016 .

[5]  Ali A. Ghorbani,et al.  Y-means: a clustering method for intrusion detection , 2003, CCECE 2003 - Canadian Conference on Electrical and Computer Engineering. Toward a Caring and Humane Technology (Cat. No.03CH37436).

[6]  Keith W. Miller,et al.  Big Data: New Opportunities New , 2013 .

[7]  Miran Brezocnik,et al.  Genetic Based Approach to Predicting the Elongation of Drawn Alloy , 2015 .

[8]  J. Alberto Espinosa,et al.  Big Data: Issues and Challenges Moving Forward , 2013, 2013 46th Hawaii International Conference on System Sciences.

[9]  Elsayed A. Sallam,et al.  A hybrid network intrusion detection framework based on random forests and weighted k-means , 2013 .

[10]  Hari Om,et al.  A hybrid system for reducing the false alarm rate of anomaly intrusion detection system , 2012, 2012 1st International Conference on Recent Advances in Information Technology (RAIT).

[11]  Yu MULTI-OBJECTIVE RESCHEDULING MODEL FOR PRODUCT COLLABORATIVE DESIGN CONSIDERING DISTURBANCE , 2014 .

[12]  G. D. Yu,et al.  Multi-Objective Rescheduling Model for Product Collaborative Design Considering Disturbance , 2014 .

[13]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[14]  Bernhard Pfahringer,et al.  Winning the KDD99 classification cup: bagged boosting , 2000, SKDD.

[15]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[16]  Jaideep Srivastava,et al.  Intrusion Detection: A Survey , 2005 .

[17]  Keith W. Miller,et al.  Big Data: New Opportunities and New Challenges [Guest editors' introduction] , 2013, Computer.

[18]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.