A Hybrid Intrusion Detection System Based on Scalable K-Means+ Random Forest and Deep Learning

Digital assets have come under various network security threats in the digital age. As a kind of security equipment to protect digital assets, intrusion detection system (IDS) is less efficient if the alert is not timely and IDS is useless if the accuracy cannot meet the requirements. Therefore, an intrusion detection model that combines machine learning with deep learning is proposed in this paper. The model uses the k-means and the random forest (RF) algorithms for the binary classification, and distributed computing of these algorithms is implemented on the Spark platform to quickly classify normal events and attack events. Then, by using the convolutional neural network (CNN), long short-term memory (LSTM), and other deep learning algorithms, the events judged as abnormal are further classified into different attack types finally. At this stage, adaptive synthetic sampling (ADASYN) is adopted to solve the unbalanced dataset. The NSL-KDD and CIS-IDS2017 datasets are used to evaluate the performance of the proposed model. The experimental results show that the proposed model has better TPR for most of attack events, faster data preprocessing speed, and potentially less training time. In particular, the accuracy of multi-target classification can reach as high as 85.24% in the NSL-KDD dataset and 99.91% in the CIC-IDS2017 dataset.

[1]  K. P. Soman,et al.  Deep Learning Approach for Intelligent Intrusion Detection System , 2019, IEEE Access.

[2]  Ali A. Ghorbani,et al.  Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization , 2018, ICISSP.

[3]  Chen Dong,et al.  Multi-dimensional feature fusion and stacking ensemble mechanism for network intrusion detection , 2021, Future Gener. Comput. Syst..

[4]  Soosan Naderi Mighan,et al.  A novel scalable intrusion detection system based on deep learning , 2020, International Journal of Information Security.

[5]  Feng Liu,et al.  A Deep Learning Approach for Network Intrusion Detection Based on NSL-KDD Dataset , 2019, 2019 IEEE 13th International Conference on Anti-counterfeiting, Security, and Identification (ASID).

[6]  Mohamed Haggag,et al.  Implementing a Deep Learning Model for Intrusion Detection on Apache Spark Platform , 2020, IEEE Access.

[7]  Yiqiang Sheng,et al.  HAST-IDS: Learning Hierarchical Spatial-Temporal Features Using Deep Neural Networks to Improve Intrusion Detection , 2018, IEEE Access.

[8]  Xiu-Shen Wei,et al.  Mask-CNN: Localizing parts and selecting descriptors for fine-grained bird species categorization , 2018, Pattern Recognit..

[9]  Lalu Banoth,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2017 .

[10]  Heba F. Eid,et al.  Hybrid Intelligent Intrusion Detection Scheme , 2011 .

[11]  Lina Yao,et al.  Automatic Device Classification from Network Traffic Streams of Internet of Things , 2018, 2018 IEEE 43rd Conference on Local Computer Networks (LCN).

[12]  Prajoona Valsalan,et al.  Retracted: Intrusion detection in cloud environment using hybrid genetic algorithm and back propagation neural network , 2020, Int. J. Commun. Syst..

[13]  I. Sumaiya Thaseen,et al.  Integrated Intrusion Detection Model Using Chi-Square Feature Selection and Ensemble of Classifiers , 2018, Arabian Journal for Science and Engineering.

[14]  Md. Rezaul Karim,et al.  A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network , 2019, Symmetry.

[15]  K. P. Soman,et al.  Applying convolutional neural network for network intrusion detection , 2017, 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[16]  Tankut Acarman,et al.  A deep learning method to detect network intrusion through flow‐based features , 2018, Int. J. Netw. Manag..

[17]  Leandros A. Maglaras,et al.  A novel intrusion detection method based on OCSVM and K-means recursive clustering , 2015, EAI Endorsed Trans. Security Safety.

[18]  Ram Ratan Ahirwal,et al.  A SVM and K-means Clustering based Fast and Efficient Intrusion Detection System , 2013 .

[19]  Zahid Akhtar,et al.  KDD Cup 99 Data Sets: A Perspective on the Role of Data Sets in Network Intrusion Detection Research , 2019, Computer.

[20]  Samrat Kumar Dey,et al.  Performance Analysis of SDN-Based Intrusion Detection Model with Feature Selection Approach , 2018, IJCCI.

[21]  Yangwoo Kim,et al.  A Two-Stage Big Data Analytics Framework with Real World Applications Using Spark Machine Learning and Long Short-Term Memory Network , 2018, Symmetry.

[22]  Deris Stiawan,et al.  Attack classification of an intrusion detection system using deep learning and hyperparameter optimization , 2021, J. Inf. Secur. Appl..

[23]  Mohamed Amine Ferrag,et al.  Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study , 2020, J. Inf. Secur. Appl..

[24]  Amit D. Sagale,et al.  Hybrid Model For Intrusion Detection Using Naive Bayesian And Support Vector Machine , 2014 .

[25]  Blaž Zupan,et al.  openTSNE: a modular Python library for t-SNE dimensionality reduction and embedding , 2019, bioRxiv.

[26]  Lifeng Cao,et al.  Analysis of Multi-Types of Flow Features Based on Hybrid Neural Network for Improving Network Anomaly Detection , 2019, IEEE Access.

[27]  Stephan Hoyer,et al.  Assessing microscope image focus quality with deep learning , 2018, BMC Bioinformatics.

[28]  Hee-su Chae,et al.  Selection for efficient Intrusion Detection using Attribute Ratio , .

[29]  Ali H. Mirza,et al.  Computer network intrusion detection using sequential LSTM Neural Networks autoencoders , 2018, 2018 26th Signal Processing and Communications Applications Conference (SIU).

[30]  Carlos Becker Westphall,et al.  Hybrid approach to intrusion detection in fog-based IoT environments , 2020, Comput. Networks.

[31]  Gholamhossein Dastghaibyfard,et al.  Two-tier network anomaly detection model: a machine learning approach , 2017, Journal of Intelligent Information Systems.

[32]  Chunning Meng,et al.  Webcam-Based Eye Movement Analysis Using CNN , 2017, IEEE Access.

[33]  Muhammad Asim,et al.  DeepDetect: Detection of Distributed Denial of Service Attacks Using Deep Learning , 2019, Comput. J..

[34]  Ali Dehghantanha,et al.  A Two-Layer Dimension Reduction and Two-Tier Classification Model for Anomaly-Based Intrusion Detection in IoT Backbone Networks , 2019, IEEE Transactions on Emerging Topics in Computing.

[35]  Kehe Wu,et al.  A Novel Intrusion Detection Model for a Massive Network Using Convolutional Neural Networks , 2018, IEEE Access.

[36]  Zixue Cheng,et al.  CNN for situations understanding based on sentiment analysis of twitter data , 2017 .

[37]  Yixian Yang,et al.  Building an Effective Intrusion Detection System Using the Modified Density Peak Clustering Algorithm and Deep Belief Networks , 2019, Applied Sciences.

[38]  K. P. Soman,et al.  A Comparative Analysis of Deep Learning Approaches for Network Intrusion Detection Systems (N-IDSs): Deep Learning for N-IDSs , 2019, Int. J. Digit. Crime Forensics.

[39]  Punam Bedi,et al.  Siam-IDS: Handling class imbalance problem in Intrusion Detection Systems using Siamese Neural Network , 2020 .

[40]  Govind P. Gupta,et al.  Performance analysis of network intrusion detection schemes using Apache Spark , 2016, 2016 International Conference on Communication and Signal Processing (ICCSP).

[41]  Devesh Kumar Srivastava,et al.  Network Intrusion Detection in Big Dataset Using Spark , 2018 .

[42]  S. P. Shantharajah,et al.  A Study on NSL-KDD Dataset for Intrusion Detection System Based on Classification Algorithms , 2015 .