Deep learning methods in network intrusion detection: A survey and an objective comparison

Abstract The use of deep learning models for the network intrusion detection task has been an active area of research in cybersecurity. Although several excellent surveys cover the growing body of research on this topic, the literature lacks an objective comparison of the different deep learning models within a controlled environment, especially on recent intrusion detection datasets. In this paper, we first introduce a taxonomy of deep learning models in intrusion detection and summarize the research papers on this topic. Then we train and evaluate four key deep learning models - feed-forward neural network, autoencoder, deep belief network and long short-term memory network - for the intrusion classification task on two legacy datasets (KDD 99, NSL-KDD) and two modern datasets (CIC-IDS2017, CIC-IDS2018). Our results suggest that deep feed-forward neural networks yield desirable evaluation metrics on all four datasets in terms of accuracy, F1-score and training and inference time. The results also indicate that two popular semi-supervised learning models, autoencoders and deep belief networks do not perform better than supervised feed-forward neural networks. The implementation and the complete set of results have been released for future use by the research community. Finally, we discuss the issues in the research literature that were revealed in the survey and suggest several potential future directions for research in machine learning methods for intrusion detection.

[1]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition , 2012 .

[2]  Dong Jin,et al.  A Comparative Study of Off-Line Deep Learning Based Network Intrusion Detection , 2018, 2018 Tenth International Conference on Ubiquitous and Future Networks (ICUFN).

[3]  Ralf C. Staudemeyer,et al.  Applying long short-term memory recurrent neural networks to intrusion detection , 2015 .

[4]  Feng Jiang,et al.  Deep Learning Based Multi-Channel Intelligent Attack Detection for Data Security , 2020, IEEE Transactions on Sustainable Computing.

[5]  Andrew H. Sung,et al.  Intrusion detection using neural networks and support vector machines , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[6]  Abdolreza Mirzaei,et al.  Transfer Learning Based Intrusion Detection , 2018, 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE).

[7]  Je-Won Kang,et al.  Intrusion Detection System Using Deep Neural Network for In-Vehicle Network Security , 2016, PloS one.

[8]  Jinoh Kim,et al.  An Empirical Study on Network Anomaly Detection Using Convolutional Neural Networks , 2018, 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS).

[9]  Christian Diedrich,et al.  Accelerated deep neural networks for enhanced Intrusion Detection System , 2016, 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA).

[10]  Mansoor Alam,et al.  A Deep Learning Approach for Network Intrusion Detection System , 2016, EAI Endorsed Trans. Security Safety.

[11]  Alexander Binder,et al.  Unmasking Clever Hans predictors and assessing what machines really learn , 2019, Nature Communications.

[12]  Yang Yu,et al.  A Hybrid Spectral Clustering and Deep Neural Network Ensemble Algorithm for Intrusion Detection in Sensor Networks , 2016, Sensors.

[13]  Jin Wei,et al.  Real-Time Detection of False Data Injection Attacks in Smart Grid: A Deep Learning-Based Intelligent Mechanism , 2017, IEEE Transactions on Smart Grid.

[14]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[15]  Casey Kneale,et al.  Semisupervised Adversarial Neural Networks for Cyber Security Transfer Learning , 2019, ArXiv.

[16]  Jim A. Simpson,et al.  Network Traffic Anomaly Detection Using Recurrent Neural Networks , 2018, ArXiv.

[17]  Vijay Varadharajan,et al.  A Detailed Investigation and Analysis of Using Machine Learning Techniques for Intrusion Detection , 2019, IEEE Communications Surveys & Tutorials.

[18]  Sepp Hochreiter,et al.  Self-Normalizing Neural Networks , 2017, NIPS.

[19]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[20]  Howon Kim,et al.  Long Short Term Memory Recurrent Neural Network Classifier for Intrusion Detection , 2016, 2016 International Conference on Platform Technology and Service (PlatCon).

[21]  Naveen K. Chilamkurti,et al.  Distributed attack detection scheme using deep learning approach for Internet of Things , 2017, Future Gener. Comput. Syst..

[22]  Daniel S. Berman,et al.  A Survey of Deep Learning Methods for Cyber Security , 2019, Inf..

[23]  Georgia Sakellari,et al.  Cloud-Based Cyber-Physical Intrusion Detection for Vehicles Using Deep Learning , 2018, IEEE Access.

[24]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[25]  Tijmen Tieleman,et al.  Training restricted Boltzmann machines using approximations to the likelihood gradient , 2008, ICML '08.

[26]  Fan Zhang,et al.  An Intrusion Detection System Using a Deep Neural Network With Gated Recurrent Units , 2018, IEEE Access.

[27]  Md Zahangir Alom,et al.  Intrusion detection using deep belief networks , 2015, 2015 National Aerospace and Electronics Conference (NAECON).

[28]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[29]  Robert C. Atkinson,et al.  Shallow and Deep Networks Intrusion Detection System: A Taxonomy and Survey , 2017, ArXiv.

[30]  Chunhua Wang,et al.  Machine Learning and Deep Learning Methods for Cybersecurity , 2018, IEEE Access.

[31]  Sang Hyun Kim,et al.  Method of intrusion detection using deep neural network , 2017, 2017 IEEE International Conference on Big Data and Smart Computing (BigComp).

[32]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[33]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[34]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Naveen K. Chilamkurti,et al.  Leveraging LSTM Networks for Attack Detection in Fog-to-Things Communications , 2018, IEEE Communications Magazine.

[36]  Carla Purdy,et al.  Toward an Online Anomaly Intrusion Detection System Based on Deep Learning , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[37]  Parvez Faruki,et al.  Network Intrusion Detection for IoT Security Based on Learning Techniques , 2019, IEEE Communications Surveys & Tutorials.

[38]  Ali A. Ghorbani,et al.  Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization , 2018, ICISSP.

[39]  Alfredo De Santis,et al.  Network anomaly detection with the restricted Boltzmann machine , 2013, Neurocomputing.

[40]  Jinoh Kim,et al.  A survey of deep learning-based network anomaly detection , 2017, Cluster Computing.

[41]  Nhien-An Le-Khac,et al.  Collective Anomaly Detection Based on Long Short-Term Memory Recurrent Neural Networks , 2016, FDSE.

[42]  Anamika Yadav,et al.  Performance analysis of NSL-KDD dataset using ANN , 2015, 2015 International Conference on Signal Processing and Communication Engineering Systems.

[43]  Jonathan M. Spring,et al.  Recognition Strategies: Intrusion Detection and Prevention , 2014 .

[44]  Naveen K. Chilamkurti,et al.  Survey on SDN based network intrusion detection system using machine learning approaches , 2018, Peer-to-Peer Networking and Applications.

[45]  Erhan Guven,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2016, IEEE Communications Surveys & Tutorials.

[46]  Wei-Yang Lin,et al.  Intrusion detection by machine learning: A review , 2009, Expert Syst. Appl..

[47]  Robert C. Atkinson,et al.  Threat analysis of IoT networks using artificial neural network intrusion detection system , 2016, 2016 International Symposium on Networks, Computers and Communications (ISNCC).

[48]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[49]  Naveen K. Chilamkurti,et al.  Deep Learning: The Frontier for Distributed Attack Detection in Fog-to-Things Computing , 2018, IEEE Communications Magazine.

[50]  Yi Zeng,et al.  $Deep-Full-Range$ : A Deep Learning Based Network Encrypted Traffic Classification and Intrusion Detection Framework , 2019, IEEE Access.

[51]  K. P. Soman,et al.  Deep Learning Approach for Intelligent Intrusion Detection System , 2019, IEEE Access.

[52]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[53]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[54]  Zheng Wang,et al.  Deep Learning-Based Intrusion Detection With Adversaries , 2018, IEEE Access.

[55]  Pierre Baldi,et al.  Autoencoders, Unsupervised Learning, and Deep Architectures , 2011, ICML Unsupervised and Transfer Learning.

[56]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[58]  Mounir Ghogho,et al.  Deep learning approach for Network Intrusion Detection in Software Defined Networking , 2016, 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM).

[59]  Ling Gao,et al.  An Intrusion Detection Model Based on Deep Belief Networks , 2014 .

[60]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[61]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[62]  Qi Shi,et al.  A Deep Learning Approach to Network Intrusion Detection , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[63]  Andreas Hotho,et al.  A Survey of Network-based Intrusion Detection Data Sets , 2019, Comput. Secur..

[64]  Nathalie Japkowicz,et al.  Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[65]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[66]  Xiaojiang Du,et al.  A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security , 2018, IEEE Communications Surveys & Tutorials.

[67]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[68]  Sachin Shetty,et al.  Transfer learning for detecting unknown network attacks , 2019, EURASIP Journal on Information Security.

[69]  Chong Wang,et al.  Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin , 2015, ICML.

[70]  Mohamed Amine Ferrag,et al.  Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study , 2020, J. Inf. Secur. Appl..

[71]  Dewan Md. Farid,et al.  Application of Machine Learning Approaches in Intrusion Detection System: A Survey , 2015 .

[72]  Ming Zhu,et al.  Malware traffic classification using convolutional neural network for representation learning , 2017, 2017 International Conference on Information Networking (ICOIN).

[73]  Yuancheng Li,et al.  A Hybrid Malicious Code Detection Method based on Deep Learning , 2015 .

[74]  Howon Kim,et al.  An Effective Intrusion Detection Classifier Using Long Short-Term Memory with Gradient Descent Optimization , 2017, 2017 International Conference on Platform Technology and Service (PlatCon).

[75]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[76]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..