A hierarchical network intrusion detection model based on unsupervised clustering

In the complex Internet of Things(IoT) environment, the security of digital ecosystems connected to the Web is guaranteed by network Intrusion Detection Systems (IDS). So far, the existing unsupervised learning methods extract the features of network traffic at the overall level, which cannot guarantee real-time network intrusion detection. To fill this gap, we propose a hierarchical network intrusion detection model based on unsupervised clustering, which is realized by combining Deep Auto-Encoder(DAE) and Gaussian Mixture Model (GMM). For new network traffic, essential features are extracted based on the first few packets, which guarantee real-time network intrusion detection. The proposed model adopts a two-layer hierarchical structure. The first layer namely the anomaly detection sub-model is based on DAGMM, which can detect abnormal traffic in real-time. The second layer namely the attack recognition sub-model identifies the attack categories of abnormal traffic detected by the anomaly detection sub-model, and getting rid of the difficulty of reconstructing abnormal traffic in DAE. The experimental results on the CICIDS2017 dataset show that the proposed model has better performance in detecting abnormal traffic and identifying the attack categories of abnormal traffic than other existing unsupervised methods.

[1]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[2]  Yinhui Li,et al.  An efficient intrusion detection system based on support vector machines and gradually feature removal method , 2012, Expert Syst. Appl..

[3]  Bruno Volckaert,et al.  Unsupervised Machine Learning Techniques for Network Intrusion Detection on Modern Data , 2020, 2020 4th Cyber Security in Networking Conference (CSNet).

[4]  Changyong Li,et al.  I2DS: Interpretable Intrusion Detection System Using Autoencoder and Additive Tree , 2021, Secur. Commun. Networks.

[5]  Ali A. Ghorbani,et al.  Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization , 2018, ICISSP.

[6]  Fang Dong,et al.  A hybrid unsupervised clustering-based anomaly detection method , 2021, Tsinghua Science and Technology.

[7]  Muhammad Fahim,et al.  A Lightweight Deep Autoencoder-Based Approach for Unsupervised Anomaly Detection , 2019, 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA).

[8]  Mei Song,et al.  PCCN: Parallel Cross Convolutional Neural Network for Abnormal Network Traffic Flows Detection in Multi-Class Imbalanced Network Traffic Flows , 2019, IEEE Access.

[9]  Hans-Peter Kriegel,et al.  A survey on unsupervised outlier detection in high‐dimensional numerical data , 2012, Stat. Anal. Data Min..

[10]  Md. Ahsanul Kabir,et al.  Unsupervised Learning for Network Flow Based Anomaly Detection in the Era of Deep Learning , 2020, 2020 IEEE Sixth International Conference on Big Data Computing Service and Applications (BigDataService).

[11]  Wooju Kim,et al.  Unsupervised learning approach for network intrusion detection system using autoencoders , 2019, The Journal of Supercomputing.

[12]  Verónica Bolón-Canedo,et al.  Performance evaluation of unsupervised techniques in cyber-attack anomaly detection , 2019, Journal of Ambient Intelligence and Humanized Computing.

[13]  Juntae Kim,et al.  The Anomaly Detection by Using DBSCAN Clustering with Multiple Parameters , 2011, 2011 International Conference on Information Science and Applications.

[14]  Ali A. Ghorbani,et al.  Towards a Reliable Intrusion Detection Benchmark Dataset , 2017 .