A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network

With the rapid advancements of ubiquitous information and communication technologies, a large number of trustworthy online systems and services have been deployed. However, cybersecurity threats are still mounting. An intrusion detection (ID) system can play a significant role in detecting such security threats. Thus, developing an intelligent and accurate ID system is a non-trivial research problem. Existing ID systems that are typically used in traditional network intrusion detection system often fail and cannot detect many known and new security threats, largely because those approaches are based on classical machine learning methods that provide less focus on accurate feature selection and classification. Consequently, many known signatures from the attack traffic remain unidentifiable and become latent. Furthermore, since a massive network infrastructure can produce large-scale data, these approaches often fail to handle them flexibly, hence are not scalable. To address these issues and improve the accuracy and scalability, we propose a scalable and hybrid IDS, which is based on Spark ML and the convolutional-LSTM (Conv-LSTM) network. This IDS is a two-stage ID system: the first stage employs the anomaly detection module, which is based on Spark ML. The second stage acts as a misuse detection module, which is based on the Conv-LSTM network, such that both global and local latent threat signatures can be addressed. Evaluations of several baseline models in the ISCX-UNB dataset show that our hybrid IDS can identify network misuses accurately in 97.29% of cases and outperforms state-of-the-art approaches during 10-fold cross-validation tests.

[1]  Ling Gao,et al.  An Intrusion Detection Model Based on Deep Belief Networks , 2014 .

[2]  Amit Sethi,et al.  Abnormal Event Detection on BMTT-PETS 2017 Surveillance Challenge , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[4]  Ali A. Ghorbani,et al.  Toward developing a systematic approach to generate benchmark datasets for intrusion detection , 2012, Comput. Secur..

[5]  Howon Kim,et al.  An Effective Intrusion Detection Classifier Using Long Short-Term Memory with Gradient Descent Optimization , 2017, 2017 International Conference on Platform Technology and Service (PlatCon).

[6]  Manu Bijone,et al.  A Survey on Secure Network: Intrusion Detection & Prevention Approaches , 2016 .

[7]  Gürsel Serpen,et al.  Application of Machine Learning Algorithms to KDD Intrusion Detection Dataset within Misuse Detection Context , 2003, MLMTA.

[8]  Wail S. Elkilani,et al.  A hybrid approach for efficient anomaly detection using metaheuristic methods , 2014, Journal of advanced research.

[9]  Yue Wu,et al.  A New Intrusion Detection System Based on KNN Classification Algorithm in Wireless Sensor Network , 2014, J. Electr. Comput. Eng..

[10]  Yangwoo Kim,et al.  A Two-Stage Big Data Analytics Framework with Real World Applications Using Spark Machine Learning and Long Short-Term Memory Network , 2018, Symmetry.

[11]  Emin Anarim,et al.  An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks , 2005, Expert Syst. Appl..

[12]  Sami Bourouis,et al.  A Real Time Adaptive Intrusion Detection Alert Classifier for High Speed Networks , 2013, 2013 IEEE 12th International Symposium on Network Computing and Applications.

[13]  Xi Wang,et al.  Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification , 2015, ACM Multimedia.

[14]  Siyang Zhang,et al.  A novel hybrid KPCA and SVM with GA model for intrusion detection , 2014, Appl. Soft Comput..

[15]  Fan Zhang,et al.  An Intrusion Detection System Using a Deep Neural Network With Gated Recurrent Units , 2018, IEEE Access.

[16]  Mohsen Kahani,et al.  Deep Learning Based Latent Feature Extraction for Intrusion Detection , 2018, Electrical Engineering (ICEE), Iranian Conference on.

[17]  Gulshan Kumar,et al.  Design of an Evolutionary Approach for Intrusion Detection , 2013, TheScientificWorldJournal.

[18]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[19]  Mohammad Zulkernine,et al.  Random-Forests-Based Network Intrusion Detection Systems , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[20]  Ralf C. Staudemeyer,et al.  Extracting salient features for network intrusion detection using machine learning methods , 2014, South Afr. Comput. J..

[21]  Nur Izura Udzir,et al.  Anomaly-based intrusion detection through K-means clustering and naives bayes classification , 2013 .

[22]  K. P. Soman,et al.  Applying convolutional neural network for network intrusion detection , 2017, 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[23]  Ralf C. Staudemeyer,et al.  Evaluating performance of long short-term memory recurrent neural networks on intrusion detection data , 2013, SAICSIT '13.

[24]  Aida Mustapha,et al.  Effective Dimensionality Reduction of Payload-Based Anomaly Detection in TMAD Model for HTTP Payload , 2016, KSII Trans. Internet Inf. Syst..

[25]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[26]  John S. Baras,et al.  On the Mitigation of Interference Imposed by Intruders in Passive RFID Networks , 2016, GameSec.

[27]  Mansoor Alam,et al.  A Deep Learning Approach for Network Intrusion Detection System , 2016, EAI Endorsed Trans. Security Safety.

[28]  Jürgen Schmidhuber,et al.  Learning Precise Timing with LSTM Recurrent Networks , 2003, J. Mach. Learn. Res..

[29]  Norliza Katuk,et al.  Oving K-Means Clustering using discretization technique in Network Intrusion Detection System , 2016, 2016 3rd International Conference on Computer and Information Sciences (ICCOINS).

[30]  Andrew H. Sung,et al.  Intrusion detection using an ensemble of intelligent paradigms , 2005, J. Netw. Comput. Appl..

[31]  Stefan Axelsson,et al.  Intrusion Detection Systems: A Survey and Taxonomy , 2002 .

[32]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[33]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[34]  William L. Fithen,et al.  State of the Practice of Intrusion Detection Technologies , 2000 .

[35]  Dietrich Rebholz-Schuhmann,et al.  Recurrent Deep Embedding Networks for Genotype Clustering and Ethnicity Prediction , 2018, ArXiv.

[36]  K. V. N. Sunitha,et al.  Effective discriminant function for intrusion detection using SVM , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[37]  Gisung Kim,et al.  A novel hybrid intrusion detection method integrating anomaly detection with misuse detection , 2014, Expert Syst. Appl..

[38]  M. A. Jabbar,et al.  Random Forest Modeling for Network Intrusion Detection System , 2016 .

[39]  Fakhri Karray,et al.  Features Selection for Intrusion Detection Systems Based on Support Vector Machines , 2009, 2009 6th IEEE Consumer Communications and Networking Conference.

[40]  Kamel Mohamed Faraoun,et al.  Neural Networks Learning Improvement using the K-Means Clustering Algorithm to Detect Network Intrusions , 2007 .

[41]  Robert C. Atkinson,et al.  Shallow and Deep Networks Intrusion Detection System: A Taxonomy and Survey , 2017, ArXiv.

[42]  Ralf C. Staudemeyer,et al.  Applying long short-term memory recurrent neural networks to intrusion detection , 2015 .

[43]  Sanjay Sharma,et al.  Intrusion Detection System: A Review , 2015 .

[44]  Xiangjian He,et al.  Detection of Denial-of-Service Attacks Based on Computer Vision Techniques , 2015, IEEE Transactions on Computers.

[45]  Noorhaniza Wahid,et al.  A hybrid network intrusion detection system using simplified swarm optimization (SSO) , 2012, Appl. Soft Comput..

[46]  Mehdi MORADI,et al.  A Neural Network Based System for Intrusion Detection and Classification of Attacks , 2004 .

[47]  Anamika Yadav,et al.  Performance analysis of NSL-KDD dataset using ANN , 2015, 2015 International Conference on Signal Processing and Communication Engineering Systems.

[48]  Ramiro Gonçalves,et al.  Intrusion detection systems in Internet of Things: A literature review , 2018, 2018 13th Iberian Conference on Information Systems and Technologies (CISTI).

[49]  Yuanliu Liu,et al.  Video-based emotion recognition using CNN-RNN and C3D hybrid networks , 2016, ICMI.

[50]  Rezaul Karim,et al.  Deep learning with TensorFlow : explore neural networks and build intelligent systems with Python , 2018 .