Multi-layer intrusion detection system with ExtraTrees feature selection, extreme learning machine ensemble, and softmax aggregation

Recent advances in intrusion detection systems based on machine learning have indeed outperformed other techniques, but struggle with detecting multiple classes of attacks with high accuracy. We propose a method that works in three stages. First, the ExtraTrees classifier is used to select relevant features for each type of attack individually for each (ELM). Then, an ensemble of ELMs is used to detect each type of attack separately. Finally, the results of all ELMs are combined using a softmax layer to refine the results and increase the accuracy further. The intuition behind our system is that multi-class classification is quite difficult compared to binary classification. So, we divide the multi-class problem into multiple binary classifications. We test our method on the UNSW and KDDcup99 datasets. The results clearly show that our proposed method is able to outperform all the other methods, with a high margin. Our system is able to achieve 98.24% and 99.76% accuracy for multi-class classification on the UNSW and KDDcup99 datasets, respectively. Additionally, we use the weighted extreme learning machine to alleviate the problem of imbalance in classification of attacks, which further boosts performance. Lastly, we implement the ensemble of ELMs in parallel using GPUs to perform intrusion detection in real time.

[1]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[2]  Mark D. McDonnell,et al.  Deep extreme learning machines: supervised autoencoding architecture for classification , 2016, Neurocomputing.

[3]  Zhe Yang,et al.  Soft extreme learning machine for fault detection of aircraft engine , 2019, Aerospace Science and Technology.

[4]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[5]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[6]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[7]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[8]  Keqin Li,et al.  Extreme Learning Machine and Its Applications in Big Data Processing , 2017 .

[9]  Magnus Westerlund,et al.  Using extreme learning machine for intrusion detection in a big data environment , 2014, AISec '14.

[10]  P. Bahr,et al.  Sampling: Theory and Applications , 2020, Applied and Numerical Harmonic Analysis.

[11]  Bernie Mulgrew,et al.  IEEE International Joint Conference on Neural Networks , 1999 .

[12]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[13]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[14]  H. N. Suresh,et al.  Medical Image Segmentation Based on Extreme Learning Machine Algorithm in Kernel Fuzzy C-Means Using Artificial Bee Colony Method , 2018, International Journal of Intelligent Engineering and Systems.

[15]  Mohamed Idhammad LabSIV.  DoS Detection Method based on Artificial Neural Networks , 2017 .

[16]  Shahaboddin Shamshirband,et al.  Using SVM-RSM and ELM-RSM Approaches for Optimizing the Production Process of Methyl and Ethyl Esters , 2018, Energies.

[17]  Jill Slay,et al.  Anomaly Detection System Using Beta Mixture Models and Outlier Detection , 2018 .

[18]  Ainuddin Wahid Abdul Wahab,et al.  Source camera identification: a distributed computing approach using Hadoop , 2017, Journal of Cloud Computing.

[19]  Salah El Hadaj,et al.  A Two-Stage Classifier Approach using RepTree Algorithm for Network Intrusion Detection , 2017 .

[20]  T. Greville,et al.  Some Applications of the Pseudoinverse of a Matrix , 1960 .

[21]  Jian Zhang,et al.  Deep Extreme Learning Machine and Its Application in EEG Classification , 2015 .

[22]  Kwok-wing Chau,et al.  A Novel Detection Algorithm to Identify False Data Injection Attacks on Power System State Estimation , 2019, Energies.

[23]  Sanjay Ghemawat,et al.  MapReduce: simplified data processing on large clusters , 2008, CACM.

[24]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[25]  Mohammed Samaka,et al.  Machine Learning for Anomaly Detection and Categorization in Multi-Cloud Environments , 2017, 2017 IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud).

[26]  Tao Tang,et al.  The QoS Indicators Analysis of Integrated EUHT Wireless Communication System Based on Urban Rail Transit in High-Speed Scenario , 2018, Wirel. Commun. Mob. Comput..

[27]  Ping-Lang Yen,et al.  Engineering Applications of Intelligent Monitoring and Control 2014 , 2013 .

[28]  Ali Miri,et al.  Using the Extreme Learning Machine (ELM) technique for heart disease diagnosis , 2015, 2015 IEEE Canada International Humanitarian Technology Conference (IHTC2015).

[29]  Robert McLeod,et al.  A Multi-agent Based Cognitive Approach to Unsupervised Feature Extraction and Classification for Network Intrusion Detection , 2017 .

[30]  Kangfeng Zheng,et al.  Improving the Classification Effectiveness of Intrusion Detection by Using Improved Conditional Variational AutoEncoder and Deep Neural Network , 2019, Sensors.

[31]  Stephen S. Yau,et al.  Attack Detection in Cloud Infrastructures Using Artificial Neural Network with Genetic Feature Selection , 2016, 2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech).

[32]  Yoshua Bengio,et al.  Série Scientifique Scientific Series Incorporating Second-order Functional Knowledge for Better Option Pricing Incorporating Second-order Functional Knowledge for Better Option Pricing , 2022 .

[33]  Muhammad Salman Khan,et al.  Multiscale Hebbian neural network for cyber threat detection , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[34]  Pierre Courrieu,et al.  Fast Computation of Moore-Penrose Inverse Matrices , 2008, ArXiv.

[35]  Mark D. McDonnell,et al.  Deep Extreme Learning Machines for Classification , 2015 .

[36]  Yu Tsao,et al.  Experimental Study on Extreme Learning Machine Applications for Speech Enhancement , 2017, IEEE Access.

[37]  Adriano L. I. de Oliveira,et al.  Extreme Learning Machines for Intrusion Detection Systems , 2012, ICONIP.

[38]  Chi Cheng,et al.  Extreme learning machines for intrusion detection , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[39]  Nour Moustafa,et al.  UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set) , 2015, 2015 Military Communications and Information Systems Conference (MilCIS).

[40]  Bernard Ghanem,et al.  Representation learning with deep extreme learning machines for efficient image set classification , 2016, Neural Computing and Applications.

[41]  Karim Afdel,et al.  Semi-supervised machine learning approach for DDoS detection , 2018, Applied Intelligence.

[42]  Amr M. T. Ali-Eldin,et al.  A multi-layer perceptron approach for flow-based anomaly detection , 2017, 2017 International Symposium on Networks, Computers and Communications (ISNCC).

[43]  Vijay Varadharajan,et al.  NvCloudIDS: A security architecture to detect intrusions at network and virtualization layer in cloud environment , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[44]  Kwok-wing Chau,et al.  Effect of river flow on the quality of estuarine and coastal waters using machine learning models , 2018 .

[45]  Hossein Gharaee,et al.  A new feature selection IDS based on genetic algorithm and SVM , 2016, 2016 8th International Symposium on Telecommunications (IST).

[46]  Stephen D. Bay,et al.  The UCI KDD archive of large data sets for data mining research and experimentation , 2000, SKDD.

[47]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[48]  Ken Ferens,et al.  Network Intrusion Detection Using Machine Learning , 2016 .

[49]  Lei Yang,et al.  Sample Selected Extreme Learning Machine Based Intrusion Detection in Fog Computing and MEC , 2018, Wirel. Commun. Mob. Comput..

[50]  Mounir Ghogho,et al.  Deep Recurrent Neural Network for Intrusion Detection in SDN-based Networks , 2018, 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft).

[51]  Shie Mannor,et al.  The cross entropy method for classification , 2005, ICML.

[52]  Ismail Boumhidi,et al.  Extreme learning machine for fault detection and isolation in wind turbine , 2016, 2016 International Conference on Electrical and Information Technologies (ICEIT).

[53]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.