An effective NIDS framework based on a comprehensive survey of feature optimization and classification techniques

The technological advancement leads to an increase in the usage of the Internet with many applications and connected devices. This increased network size causes increased complexity and creating rooms for the attackers to explore and exploit vulnerabilities to carry out various attacks. As a result upsurge of network attacks can be realized in recent years and is diversified, which can be affirmed by the admittance of various organizations. Varieties of intrusion detection systems (IDSs) have been designed and proposed to tackle such issues based on the misuse-based, anomaly based, and sometimes hybrid techniques. The high rate of network data generation and its enormous volume makes it challenging for IDSs to maintain their efficacy and reliability. This paper discusses a comprehensive understanding of IDS types, six benchmark network datasets, high distributed dimensionality reduction techniques, and classification approaches based on machine learning and deep learning for intrusion detection with their importance to ascertain the efficacy and reliability of IDSs. Furthermore, based on the literature review, a general framework for NIDS has been proposed. At last model for network IDS (NIDS) is designed by following the proposed framework. Achieved accuracy and detection rate of the proposed NIDS model on the UNSW-NB15 dataset are 98.11% and 97.81%, respectively, and achieving better performance than other approaches comparatively.

[1]  Rajanikanth Aluvalu,et al.  RFAODE: A Novel Ensemble Intrusion Detection System , 2017 .

[2]  Marco Vannucci,et al.  A Fuzzy System for Combining Filter Features Selection Methods , 2016, International Journal of Fuzzy Systems.

[3]  Nikita Joshi,et al.  A Review Paper on Feature Selection Methodologies and Their Applications , 2014 .

[4]  Ling Gao,et al.  An Intrusion Detection Model Based on Deep Belief Networks , 2014 .

[5]  Xiangjian He,et al.  Building an Intrusion Detection System Using a Filter-Based Feature Selection Algorithm , 2016, IEEE Transactions on Computers.

[6]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[7]  Shahram Babaie,et al.  A hybrid intrusion detection system based on ABC-AFS algorithm for misuse and anomaly detection , 2018, Comput. Networks.

[8]  Rui Zhang,et al.  Model of the intrusion detection system based on the integration of spatial-temporal features , 2020, Comput. Secur..

[9]  Albert Y. Zomaya,et al.  En-ABC: An ensemble artificial bee colony based anomaly detection scheme for cloud environment , 2020, J. Parallel Distributed Comput..

[10]  Mohamed Rida,et al.  A Hybrid Optimization Framework Based on Genetic Algorithm and Simulated Annealing Algorithm to Enhance Performance of Anomaly Network Intrusion Detection System Based on BP Neural Network , 2018, 2018 International Symposium on Advanced Electrical and Communication Technologies (ISAECT).

[11]  Praveen Kumar Reddy Maddikunta,et al.  A Novel PCA-Firefly Based XGBoost Classification Model for Intrusion Detection in Networks Using GPU , 2020, Electronics.

[12]  Francesco Carlo Morabito,et al.  A novel statistical analysis and autoencoder driven intelligent intrusion detection approach , 2020, Neurocomputing.

[13]  Vern Paxson,et al.  Measuring Pay-per-Install: The Commoditization of Malware Distribution , 2011, USENIX Security Symposium.

[14]  Pankaj Kumar Keserwani,et al.  A smart anomaly-based intrusion detection system for the Internet of Things (IoT) network using GWO–PSO–RF model , 2021, Journal of Reliable Intelligent Environments.

[15]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[16]  Ming Zhu,et al.  End-to-end encrypted traffic classification with one-dimensional convolution neural networks , 2017, 2017 IEEE International Conference on Intelligence and Security Informatics (ISI).

[17]  Farrukh Aslam Khan,et al.  TSDL: A Two-Stage Deep Learning Model for Efficient Network Intrusion Detection , 2019, IEEE Access.

[18]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[19]  Nurdan Akhan Baykan,et al.  Feature selection in network intrusion detection using metaheuristic algorithms , 2018 .

[20]  Reazul Kabir,et al.  A Network Intrusion Detection Framework based on Bayesian Network using Wrapper Approach , 2017 .

[21]  Kuinam J. Kim,et al.  A feature selection approach to find optimal feature subsets for the network intrusion detection system , 2015, Cluster Computing.

[22]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[23]  Mei-Ling Shyu,et al.  A Survey on Deep Learning , 2018, ACM Comput. Surv..

[24]  Aiko Pras,et al.  An Overview of IP Flow-Based Intrusion Detection , 2010, IEEE Communications Surveys & Tutorials.

[25]  Manas Ranjan Patra,et al.  Fuzzy Rough Classification Models for Network Intrusion Detection , 2016 .

[26]  Félix J. García Clemente,et al.  A Self-Adaptive Deep Learning-Based System for Anomaly Detection in 5G Networks , 2018, IEEE Access.

[27]  Yang Yu,et al.  Network Intrusion Detection through Stacking Dilated Convolutional Autoencoders , 2017, Secur. Commun. Networks.

[28]  Hadis Karimipour,et al.  Cyber intrusion detection by combined feature selection algorithm , 2019, J. Inf. Secur. Appl..

[29]  Ahmad Sharieh,et al.  A feature selection algorithm for intrusion detection system based on Pigeon Inspired Optimizer , 2020 .

[30]  Antonio Martínez-Álvarez,et al.  Feature selection by multi-objective optimisation: Application to network anomaly detection by hierarchical self-organising maps , 2014, Knowl. Based Syst..

[31]  Patrick Siarry,et al.  A survey on optimization metaheuristics , 2013, Inf. Sci..

[32]  Salwani Abdullah,et al.  Hybridizing firefly algorithms with a probabilistic neural network for solving classification problems , 2015, Appl. Soft Comput..

[33]  I. Sumaiya Thaseen,et al.  Analysis on Improving the Performance of Machine Learning Models Using Feature Selection Technique , 2018, ISDA.

[34]  Yasmen Wahba,et al.  Improving the Performance of Multi-class Intrusion Detection Systems using Feature Reduction , 2015, ArXiv.

[35]  Vijay Varadharajan,et al.  PSI-NetVisor: Program semantic aware intrusion detection at network and hypervisor layer in cloud , 2017, J. Intell. Fuzzy Syst..

[36]  Chirag Modi,et al.  Designing an efficient security framework for detecting intrusions in virtual network of cloud computing , 2019, Comput. Secur..

[37]  S. Sibi Chakkaravarthy,et al.  Complex Event Processing based Hybrid Intrusion Detection System , 2015, 2015 3rd International Conference on Signal Processing, Communication and Networking (ICSCN).

[38]  Tanja Zseby,et al.  Analysis of network traffic features for anomaly detection , 2014, Machine Learning.

[39]  Seyed Mojtaba Hosseini Bamakan,et al.  Ramp loss K-Support Vector Classification-Regression; a robust and sparse multi-class approach to the intrusion detection problem , 2017, Knowl. Based Syst..

[40]  Chaouki Khammassi,et al.  A NSGA2-LR wrapper approach for feature selection in network intrusion detection , 2020, Comput. Networks.

[41]  Ditipriya Sinha,et al.  UIDS: a unified intrusion detection system for IoT environment , 2019, Evol. Intell..

[42]  Yasir Hamid,et al.  Feature selection techniques for intrusion detection using non-bio-inspired and bio-inspired optimization algorithms , 2017, Journal of Communications and Information Networks.

[43]  Ali Bou Nassif,et al.  Dimensionality reduction with IG-PCA and ensemble classifier for network intrusion detection , 2019, Comput. Networks.

[44]  Xianbin Wang,et al.  Machine learning techniques for intrusion detection on public dataset , 2016, 2016 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[45]  Harish Kumar,et al.  An intrusion detection system using network traffic profiling and online sequential extreme learning machine , 2015, Expert Syst. Appl..

[46]  Ditipriya Sinha,et al.  An integrated rule based intrusion detection system: analysis on UNSW-NB15 data set and the real time online dataset , 2019, Cluster Computing.

[47]  Mohamed Rida,et al.  Novel Framework Based on Genetic Algorithm and Simulated Annealing Algorithm for Optimization of BP Neural Network Applied to Network IDS , 2018 .

[48]  Sule Yildirim Yayilgan,et al.  A Novel Hybrid IDS Based on Modified NSGAII-ANN and Random Forest , 2020, Electronics.

[49]  Luiz Eduardo Soares de Oliveira,et al.  Toward a reliable anomaly-based intrusion detection in real-world environments , 2017, Comput. Networks.

[50]  Sanmeet Kaur,et al.  Extreme Gradient Boosting Based Tuning for Classification in Intrusion Detection Systems , 2018 .

[51]  Nasser Yazdani,et al.  Mutual information-based feature selection for intrusion detection systems , 2011, J. Netw. Comput. Appl..

[52]  Xinggao Liu,et al.  Melt index prediction by least squares support vector machines with an adaptive mutation fruit fly optimization algorithm , 2015 .

[53]  Donato Malerba,et al.  A Comparative Analysis of Methods for Pruning Decision Trees , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Ali Dehghantanha,et al.  Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing , 2016, EURASIP Journal on Wireless Communications and Networking.

[55]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[56]  Shadi Aljawarneh,et al.  Anomaly-based intrusion detection system through feature selection analysis and building hybrid efficient model , 2017, J. Comput. Sci..

[57]  Georgios Kambourakis,et al.  Dendron : Genetic trees driven rule induction for network intrusion detection systems , 2018, Future Gener. Comput. Syst..

[58]  Alejandro Zunino,et al.  An empirical comparison of botnet detection methods , 2014, Comput. Secur..

[59]  Guang Cheng,et al.  An Efficient Network Intrusion Detection System Based on Feature Selection and Ensemble Classifier , 2019, ArXiv.

[60]  Haibo Liu,et al.  A Method for Guaranteeing Wireless Communication Based on a Combination of Deep and Shallow Learning , 2019, IEEE Access.

[61]  Jill Slay,et al.  The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set , 2016, Inf. Secur. J. A Glob. Perspect..

[62]  Hao Yu,et al.  Distributed Machine Learning on Smart-Gateway Network Towards Real-Time Indoor Data Analytics , 2017 .

[63]  Henrik Sandberg,et al.  A Survey of Physics-Based Attack Detection in Cyber-Physical Systems , 2018, ACM Comput. Surv..

[64]  Hossein Mobahi,et al.  Deep Learning via Semi-supervised Embedding , 2012, Neural Networks: Tricks of the Trade.

[65]  Michel Dagenais,et al.  A deep learning approach for proactive multi-cloud cooperative intrusion detection system , 2019, Future Gener. Comput. Syst..

[66]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[67]  Jose Miguel Puerta,et al.  A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets , 2011, Pattern Recognit. Lett..

[68]  Craig A. Knoblock,et al.  A Survey of Digital Map Processing Techniques , 2014, ACM Comput. Surv..

[69]  Giancarlo Fortino,et al.  A hybrid deep learning model for efficient intrusion detection in big data environment , 2020, Inf. Sci..

[70]  Kwangjo Kim,et al.  Deep Abstraction and Weighted Feature Selection for Wi-Fi Impersonation Detection , 2018, IEEE Transactions on Information Forensics and Security.

[71]  Naveen K. Chilamkurti,et al.  Deep Learning: The Frontier for Distributed Attack Detection in Fog-to-Things Computing , 2018, IEEE Communications Magazine.

[72]  V. S. Shankar Sriram,et al.  An efficient intrusion detection system based on hypergraph - Genetic algorithm for parameter optimization and feature selection in support vector machine , 2017, Knowl. Based Syst..

[73]  Jason J. Jung,et al.  HIDCC: A hybrid intrusion detection approach in cloud computing , 2018, Concurr. Comput. Pract. Exp..

[74]  Akhan Akbulut,et al.  Evolving deep learning architectures for network intrusion detection using a double PSO metaheuristic , 2020, Comput. Networks.

[75]  R. Vijayanand,et al.  Intrusion detection system for wireless mesh network using multiple support vector machine classifiers with genetic-algorithm-based feature selection , 2018, Comput. Secur..

[76]  Robert C. Atkinson,et al.  Threat analysis of IoT networks using artificial neural network intrusion detection system , 2016, 2016 International Symposium on Networks, Computers and Communications (ISNCC).

[77]  Miad Faezipour,et al.  Deep and Machine Learning Approaches for Anomaly-Based Intrusion Detection of Imbalanced Network Traffic , 2019, IEEE Sensors Letters.

[78]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[79]  Jie Li,et al.  DeepWindow: An Efficient Method for Online Network Traffic Anomaly Detection , 2019, 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[80]  Yuancheng Li,et al.  A Hybrid Malicious Code Detection Method based on Deep Learning , 2015 .

[81]  Ali A. Ghorbani,et al.  Characterization of Encrypted and VPN Traffic using Time-related Features , 2016, ICISSP.

[82]  M Namratha,et al.  A Comprehensive Overview of Clustering Algorithms in Pattern Recognition , 2012 .

[83]  Li Deng,et al.  A tutorial survey of architectures, algorithms, and applications for deep learning , 2014, APSIPA Transactions on Signal and Information Processing.

[84]  Tyrone. Naidoo,et al.  Unsupervised feature selection for anomaly-based network intrusion detection using cluster validity indices. , 2015 .

[85]  Yuxin Ding,et al.  Host-based intrusion detection using dynamic and static behavioral models , 2003, Pattern Recognit..

[86]  Tharam S. Dillon,et al.  CorrCorr: A feature selection method for multivariate correlation network anomaly detection techniques , 2019, Comput. Secur..

[87]  Dandan Liu,et al.  DL-IDS: a deep learning-based intrusion detection framework for securing IoT , 2019, Trans. Emerg. Telecommun. Technol..

[88]  Seyed Mojtaba Hosseini Bamakan,et al.  An effective intrusion detection framework based on MCLP/SVM optimized by time-varying chaos particle swarm optimization , 2016, Neurocomputing.

[89]  Jugal K. Kalita,et al.  MIFS-ND: A mutual information-based feature selection method , 2014, Expert Syst. Appl..

[90]  EMMANOUIL VASILOMANOLAKIS,et al.  Taxonomy and Survey of Collaborative Intrusion Detection , 2015, ACM Comput. Surv..

[91]  K. Muneeswaran,et al.  Firefly algorithm based feature selection for network intrusion detection , 2019, Comput. Secur..

[92]  Novriandi,et al.  Influence Analysis of Feature Selection to Network Intrusion Detection System Performance Using NSL-KDD Dataset , 2019, 2019 International Conference on Computer Science, Information Technology, and Electrical Engineering (ICOMITEE).

[93]  Kwangjo Kim,et al.  Improving Detection of Wi-Fi Impersonation by Fully Unsupervised Deep Learning , 2017, WISA.

[94]  Yang Yu,et al.  A Hybrid Spectral Clustering and Deep Neural Network Ensemble Algorithm for Intrusion Detection in Sensor Networks , 2016, Sensors.

[95]  Jiankun Hu,et al.  A Semantic Approach to Host-Based Intrusion Detection Systems Using Contiguousand Discontiguous System Call Patterns , 2014, IEEE Transactions on Computers.

[96]  Julio Ortega Lopera,et al.  PCA filtering and probabilistic SOM for network intrusion detection , 2015, Neurocomputing.

[97]  Ming Zhu,et al.  Malware traffic classification using convolutional neural network for representation learning , 2017, 2017 International Conference on Information Networking (ICOIN).

[98]  Gisung Kim,et al.  A novel hybrid intrusion detection method integrating anomaly detection with misuse detection , 2014, Expert Syst. Appl..

[99]  Jie Gu,et al.  An effective intrusion detection framework based on SVM with feature augmentation , 2017, Knowl. Based Syst..

[100]  Xinghuo Yu,et al.  A simple and efficient hidden Markov model scheme for host-based anomaly intrusion detection , 2009, IEEE Network.

[101]  Paul D. Yoo,et al.  DEMISe: Interpretable Deep Extraction and Mutual Information Selection Techniques for IoT Intrusion Detection , 2019, ARES.

[102]  Bayu Adhi Tama,et al.  TSE-IDS: A Two-Stage Classifier Ensemble for Intelligent Anomaly-Based Intrusion Detection System , 2019, IEEE Access.

[103]  Taufik Abrão,et al.  Network Anomaly Detection System using Genetic Algorithm and Fuzzy Logic , 2018, Expert Syst. Appl..

[104]  Eryk Dutkiewicz,et al.  Cyberattack detection in mobile cloud computing: A deep learning approach , 2017, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[105]  Je-Won Kang,et al.  Intrusion Detection System Using Deep Neural Network for In-Vehicle Network Security , 2016, PloS one.

[106]  Siyang Zhang,et al.  A novel hybrid KPCA and SVM with GA model for intrusion detection , 2014, Appl. Soft Comput..

[107]  Iftikhar Ahmad,et al.  Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000 , 2018 .

[108]  Nour Moustafa,et al.  UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set) , 2015, 2015 Military Communications and Information Systems Conference (MilCIS).

[109]  Annie George,et al.  Anomaly Detection based on Machine Learning Dimensionality Reduction using PCA and Classification using SVM , 2012 .

[110]  Jiadong Ren,et al.  Building an Effective Intrusion Detection System by Using Hybrid Data Optimization Based on Machine Learning Algorithms , 2019, Secur. Commun. Networks.

[111]  Mahdi Jafari Siavoshani,et al.  Deep packet: a novel approach for encrypted traffic classification using deep learning , 2017, Soft Computing.

[112]  Chun-Hung Richard Lin,et al.  Intrusion detection system: A comprehensive review , 2013, J. Netw. Comput. Appl..

[113]  Md Zahangir Alom,et al.  Network intrusion detection for cyber security using unsupervised deep learning approaches , 2017, 2017 IEEE National Aerospace and Electronics Conference (NAECON).

[114]  Miad Faezipour,et al.  Effective Features Selection and Machine Learning Classifiers for Improved Wireless Intrusion Detection , 2018, 2018 International Symposium on Networks, Computers and Communications (ISNCC).