Network Intrusion Detection Using Clustering and Gradient Boosting

An unauthorized activity on the network is called network intrusion and device or software application which monitors the network parameters in order to detect such an intrusion is called network intrusion detection system (NIDS). With high rise in malicious activities on the internet, it is extremely important for NIDS to quickly and correctly identify any kind of malicious activity on the network. Moreover, the system must refrain from raising false alarms in case of normal usage detected as malicious. This paper proposes use of machine learning classification algorithms - XGBoost and AdaBoost with and without clustering to train a model for NIDS. The models are trained and tested using NSL KDD dataset and the results are an improvement over the previous works related to intrusion detection on the same dataset.

[1]  Li-Yong Ren,et al.  Using data mining to discover signatures in network-based intrusion detection , 2002, Proceedings. International Conference on Machine Learning and Cybernetics.

[2]  A. G. Mustafaev,et al.  INTRUSION DETECTION SYSTEM , 2018 .

[3]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[4]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[5]  Trevor Hastie,et al.  Additive Logistic Regression : a Statistical , 1998 .

[6]  Lior Rokach,et al.  Top-down induction of decision trees classifiers - a survey , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[8]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[9]  Yong Guan,et al.  Sketch-Based Streaming PCA Algorithm for Network-Wide Traffic Anomaly Detection , 2010, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[10]  A.M. Cansian,et al.  Neural networks applied in intrusion detection systems , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[11]  Andreea Anghel,et al.  Catch It If You Can: Real-Time Network Anomaly Detection with Low False Alarm Rates , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[12]  Giovanni Vigna,et al.  NetSTAT: a network-based intrusion detection approach , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[13]  Atsushi Inoue,et al.  Support vector classifiers and network intrusion detection , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[14]  Hai Jin,et al.  A fuzzy data mining based intrusion detection model , 2004, Proceedings. 10th IEEE International Workshop on Future Trends of Distributed Computing Systems, 2004. FTDCS 2004..

[15]  Philippe Owezarski,et al.  Online and Scalable Unsupervised Network Anomaly Detection Method , 2017, IEEE Trans. Netw. Serv. Manag..

[16]  Ali A. Ghorbani,et al.  Y-means: a clustering method for intrusion detection , 2003, CCECE 2003 - Canadian Conference on Electrical and Computer Engineering. Toward a Caring and Humane Technology (Cat. No.03CH37436).

[17]  Hai Sheng Li.  An Intrusion Detection Based on Markov Model , 2011 .

[18]  Dewan Md Farid,et al.  Feature selection and intrusion classification in NSL-KDD cup 99 dataset employing SVMs , 2014, The 8th International Conference on Software, Knowledge, Information Management and Applications (SKIMA 2014).

[19]  Hui Wang,et al.  A clustering-based method for unsupervised intrusion detections , 2006, Pattern Recognit. Lett..

[20]  Nong Ye,et al.  Statistical process control for computer intrusion detection , 2001, Proceedings DARPA Information Survivability Conference and Exposition II. DISCEX'01.

[21]  Philippe Owezarski,et al.  UNADA: Unsupervised Network Anomaly Detection Using Sub-space Outliers Ranking , 2011, Networking.

[22]  Hany M. Harb,et al.  Adaboost Ensemble with Genetic Algorithm Post Optimization for Intrusion Detection , 2011 .

[23]  Shyhtsun Felix Wu,et al.  Statistical anomaly detection for link-state routing protocols , 1998, Proceedings Sixth International Conference on Network Protocols (Cat. No.98TB100256).

[24]  Manas Ranjan Patra,et al.  Discriminative multinomial Naïve Bayes for network intrusion detection , 2010, 2010 Sixth International Conference on Information Assurance and Security.

[25]  Mahmod S. Mahmod,et al.  A COMPARISON STUDY FOR INTRUSION DATABASE (KDD99, NSL-KDD) BASED ON SELF ORGANIZATION MAP (SOM) ARTIFICIAL NEURAL NETWORK , 2013 .