AI-IDS: Application of Deep Learning to Real-Time Web Intrusion Detection

Deep Learning has been widely applied to problems in detecting various network attacks. However, no cases on network security have shown applications of various deep learning algorithms in real-time services beyond experimental conditions. Moreover, owing to the integration of high-performance computing, it is necessary to apply systems that can handle large-scale traffic. Given the rapid evolution of web-attacks, we implemented and applied our Artificial Intelligence-based Intrusion Detection System (AI-IDS). We propose an optimal convolutional neural network and long short-term memory network (CNN-LSTM) model, normalized UTF-8 character encoding for Spatial Feature Learning (SFL) to adequately extract the characteristics of real-time HTTP traffic without encryption, calculating entropy, and compression. We demonstrated its excellence through repeated experiments on two public datasets (CSIC-2010, CICIDS2017) and fixed real-time data. By training payloads that analyzed true or false positives with a labeling tool, AI-IDS distinguishes sophisticated attacks, such as unknown patterns, encoded or obfuscated attacks from benign traffic. It is a flexible and scalable system that is implemented based on Docker images, separating user-defined functions by independent images. It also helps to write and improve Snort rules for signature-based IDS based on newly identified patterns. As the model calculates the malicious probability by continuous training, it could accurately analyze unknown web-attacks.

[1]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[2]  Qi Shi,et al.  A Deep Learning Approach to Network Intrusion Detection , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[3]  Jie Gu,et al.  A novel approach to intrusion detection using SVM ensemble with feature augmentation , 2019, Comput. Secur..

[4]  Yalei Ding,et al.  Intrusion Detection System for NSL-KDD Dataset Using Convolutional Neural Networks , 2018, CSAI '18.

[5]  Jiankun Hu,et al.  A holistic review of Network Anomaly Detection Systems: A comprehensive survey , 2019, J. Netw. Comput. Appl..

[6]  Aladdin Ayesh,et al.  Intelligent intrusion detection systems using artificial neural networks , 2018, ICT Express.

[7]  Xing Zhao,et al.  Intrusion Detection Algorithm Based on Convolutional Neural Network , 2018 .

[8]  Gürsel Serpen,et al.  Why machine learning algorithms fail in misuse detection on KDD intrusion detection data set , 2004, Intell. Data Anal..

[9]  Andreas Hotho,et al.  A Survey of Network-based Intrusion Detection Data Sets , 2019, Comput. Secur..

[10]  Muhammad Munwar Iqbal,et al.  Enhanced Network Anomaly Detection Based on Deep Neural Networks , 2018, IEEE Access.

[11]  Yaser Jararweh,et al.  An intrusion detection system for connected vehicles in smart cities , 2019, Ad Hoc Networks.

[12]  Kehe Wu,et al.  A Novel Intrusion Detection Model for a Massive Network Using Convolutional Neural Networks , 2018, IEEE Access.

[13]  Harish Kumar,et al.  A Reference Dataset for Network Traffic Activity Based Intrusion Detection System , 2015, Int. J. Comput. Commun. Control.

[14]  Mohamed Rida,et al.  Intelligent approach to build a Deep Neural Network based IDS for cloud environment using combination of machine learning algorithms , 2019, Comput. Secur..

[15]  Mohammad Saiful Islam Mamun,et al.  Tell Them from Me: An Encrypted Application Profiler , 2019, NSS.

[16]  Vijay Varadharajan,et al.  A Detailed Investigation and Analysis of Using Machine Learning Techniques for Intrusion Detection , 2019, IEEE Communications Surveys & Tutorials.

[17]  Todd L. Heberlein,et al.  Network intrusion detection , 1994, IEEE Network.

[18]  Asifullah Khan,et al.  Network anomaly detection using channel boosted and residual learning based deep convolutional neural network , 2019, Appl. Soft Comput..

[19]  Álvaro Herrero,et al.  Neural Analysis of HTTP Traffic for Web Attack Detection , 2015, CISIS-ICEUTE.

[20]  Jinxiong Zhao,et al.  SQL Injection Detection Based on Deep Belief Network , 2019, CSAE.

[21]  Erdogan Dogdu,et al.  Intrusion Detection Using Big Data and Deep Learning Techniques , 2019, ACM Southeast Regional Conference.

[22]  Hongyu Liu,et al.  CNN and RNN based payload classification methods for attack detection , 2019, Knowl. Based Syst..

[23]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[24]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[25]  K. P. Soman,et al.  Deep Learning Approach for Intelligent Intrusion Detection System , 2019, IEEE Access.

[26]  Burak Kantarci,et al.  On the Feasibility of Deep Learning in Sensor Network Intrusion Detection , 2019, IEEE Networking Letters.

[27]  Chunhua Wang,et al.  Machine Learning and Deep Learning Methods for Cybersecurity , 2018, IEEE Access.

[28]  Yiqiang Sheng,et al.  HAST-IDS: Learning Hierarchical Spatial-Temporal Features Using Deep Neural Networks to Improve Intrusion Detection , 2018, IEEE Access.