Review of Current Machine Learning Approaches for Anomaly Detection in Network Traffic

Due to the advance in network technologies, the number of network users is growing rapidly, which leads to the generation of large network traffic data. This large network traffic data is prone to attacks and intrusions. Therefore, the network needs to be secured and protected by detecting anomalies as well as to prevent intrusions into networks. Network security has gained attention from researchers and network laboratories. In this paper, a comprehensive survey was completed to give a broad perspective of what recently has been done in the area of anomaly detection. Newly published studies in the last five years have been investigated to explore modern techniques with future opportunities. In this regard, the related literature on anomaly detection systems in network traffic has been discussed, with a variety of typical applications such as WSNs, IoT, high-performance computing, industrial control systems (ICS), and software-defined network (SDN) environments. Finally, we underlined diverse open issues to improve the detection of anomaly systems.

[1]  Sumeet Dua,et al.  Data Mining and Machine Learning in Cybersecurity , 2011 .

[2]  Hui Tian,et al.  PCA-Based Network Traffic Anomaly Detection , 2016 .

[3]  Fadi Al-Turjman,et al.  An intrusion detection scheme based on the ensemble of discriminant classifiers , 2020, Comput. Electr. Eng..

[4]  Shikha Agrawal,et al.  Survey on Anomaly Detection using Data Mining Techniques , 2015, KES.

[5]  V. R. Josna,et al.  Traffic Verification for Network Anomaly Detection in Sensor Networks , 2016 .

[6]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[7]  Ernest Foo,et al.  Improving performance of intrusion detection system using ensemble methods and feature selection , 2018, ACSW.

[8]  Iqbal Gondal,et al.  An Anomaly Intrusion Detection System Using C5 Decision Tree Classifier , 2018, PAKDD.

[9]  Zhixin Sun,et al.  A Detection Method for Anomaly Flow in Software Defined Network , 2018, IEEE Access.

[10]  Christopher Leckie,et al.  Support vector machines resilient against training data integrity attacks , 2019, Pattern Recognit..

[11]  Janusz Kusyk,et al.  Survey on evolutionary computation methods for cybersecurity of mobile ad hoc networks , 2018, Evolutionary Intelligence.

[12]  Zhiyang Li,et al.  Detecting DDoS attacks against data center with correlation analysis , 2015, Comput. Commun..

[13]  Jie Gu,et al.  A novel approach to intrusion detection using SVM ensemble with feature augmentation , 2019, Comput. Secur..

[14]  Mamun Bin Ibne Reaz,et al.  A survey of intrusion detection systems based on ensemble and hybrid classifiers , 2017, Comput. Secur..

[15]  Samira Briongos,et al.  Anomaly Detection Using Gaussian Mixture Probability Model to Implement Intrusion Detection System , 2019, HAIS.

[16]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[17]  Zhen Lin,et al.  Network Anomaly Detection Based on Dynamic Hierarchical Clustering of Cross Domain Data , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C).

[18]  Yee Jian Chew,et al.  Decision Tree with Sensitive Pruning in Network-based Intrusion Detection System , 2020 .

[19]  Dieter Hogrefe,et al.  A Novel Semi-Supervised Adaboost Technique for Network Anomaly Detection , 2016, MSWiM.

[20]  Yu-Lin He,et al.  Fuzziness based semi-supervised learning approach for intrusion detection system , 2017, Inf. Sci..

[21]  Ajeet Rai,et al.  Optimizing a New Intrusion Detection System Using Ensemble Methods and Deep Neural Network , 2020, 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI)(48184).

[22]  Jing Long,et al.  A Security Situation Prediction Algorithm Based on HMM in Mobile Network , 2018, Wirel. Commun. Mob. Comput..

[23]  Hamid H. Jebur,et al.  Machine Learning Techniques for Anomaly Detection: An Overview , 2013 .

[24]  Dang Hai Hoang,et al.  A PCA-based method for IoT network traffic anomaly detection , 2018, 2018 20th International Conference on Advanced Communication Technology (ICACT).

[25]  Karim Afdel,et al.  Semi-supervised machine learning approach for DDoS detection , 2018, Applied Intelligence.

[26]  D. Ashok Kumar,et al.  A Novel Algorithm for Network Anomaly Detection Using Adaptive Machine Learning , 2018 .

[27]  Yi Yi Aung,et al.  An Analysis of K-means Algorithm Based Network Intrusion Detection System , 2018 .

[28]  Jasmin Kevric,et al.  An effective combining classifier approach using tree algorithms for network intrusion detection , 2017, Neural Computing and Applications.

[29]  Mario Vega-Barbas,et al.  Evaluation of Cybersecurity Data Set Characteristics for Their Applicability to Neural Networks Algorithms Detecting Cybersecurity Anomalies , 2020, IEEE Access.

[30]  Yang Shi,et al.  Analyzing and Visualizing Anomalies and Events in Time Series of Network Traffic , 2019 .

[31]  Joel J. P. C. Rodrigues,et al.  A comprehensive survey on network anomaly detection , 2018, Telecommunication Systems.

[32]  Subutai Ahmad,et al.  Unsupervised real-time anomaly detection for streaming data , 2017, Neurocomputing.

[33]  Afroja Bhuiyan Akhi,et al.  Network Intrusion Classification Employing Machine Learning: A Survey , 2019 .

[34]  K. P. Soman,et al.  Evaluation of Recurrent Neural Network and its Variants for Intrusion Detection System (IDS) , 2017, Int. J. Inf. Syst. Model. Des..

[35]  Madhu Shukla,et al.  A review on outlier detection techniques on data stream by using different approaches of K-Means algorithm , 2015, 2015 International Conference on Advances in Computer Engineering and Applications.

[36]  Min Ren,et al.  A Naive Bayesian Network Intrusion Detection Algorithm Based on Principal Component Analysis , 2015, 2015 7th International Conference on Information Technology in Medicine and Education (ITME).

[37]  Mohamed Guerroumi,et al.  Semi-supervised Statistical Approach for Network Anomaly Detection , 2016, ANT/SEIT.

[38]  Hadis Karimipour,et al.  Cyber intrusion detection by combined feature selection algorithm , 2019, J. Inf. Secur. Appl..

[39]  Nguyen Ha Duong,et al.  A semi-supervised model for network traffic anomaly detection , 2015, 2015 17th International Conference on Advanced Communication Technology (ICACT).

[40]  Hee-Kap Ahn,et al.  Statistical Similarity of Critical Infrastructure Network Traffic Based on Nearest Neighbor Distances , 2018, RAID.

[41]  Mohammad Asif,et al.  Implementation of K-Means Clustering for Intrusion Detection , 2019, International Journal of Scientific Research in Computer Science, Engineering and Information Technology.

[42]  Wondimu K. Zegeye,et al.  Multi-Layer Hidden Markov Model Based Intrusion Detection System , 2018, Mach. Learn. Knowl. Extr..

[43]  Giacomo Indiveri,et al.  Real-Time Ultra-Low Power ECG Anomaly Detection Using an Event-Driven Neuromorphic Processor , 2019, IEEE Transactions on Biomedical Circuits and Systems.

[44]  Sehun Kim,et al.  A Novel Hierarchical Detection Method for Enhancing Anomaly Detection Efficiency , 2015, 2015 International Conference on Computational Intelligence and Communication Networks (CICN).

[45]  Changzhen Hu,et al.  Network Security Situation Prediction Based on MR-SVM , 2019, IEEE Access.

[46]  M. C. Padma,et al.  A Study on Sentiment Analysis on Social Media Data , 2019 .

[47]  Eric D. Knapp,et al.  Industrial Network Security: Securing Critical Infrastructure Networks for Smart Grid, SCADA, and Other Industrial Control Systems , 2011 .

[48]  Robert C. Atkinson,et al.  Threat analysis of IoT networks using artificial neural network intrusion detection system , 2016, 2016 International Symposium on Networks, Computers and Communications (ISNCC).

[49]  B. Surendiran,et al.  Dimensionality reduction using Principal Component Analysis for network intrusion detection , 2016 .

[50]  Jingbing Li,et al.  Anomaly Detection in Wireless Sensor Networks Based on KNN , 2019, ICAIS.

[51]  S. B. Bagal,et al.  Performance Evaluation of K-means Clustering Algorithm with Various Distance Metrics , 2015 .

[52]  Austin Henslee,et al.  Using Gaussian Mixture Models to Detect Outliers in Seasonal Univariate Network Traffic , 2017, 2017 IEEE Security and Privacy Workshops (SPW).

[53]  Kehe Wu,et al.  A Novel Intrusion Detection Model for a Massive Network Using Convolutional Neural Networks , 2018, IEEE Access.

[54]  Salvatore J. Stolfo,et al.  Data Mining Approaches for Intrusion Detection , 1998, USENIX Security Symposium.

[55]  Azliza Mohd Ali,et al.  Anomalous behaviour detection using heterogeneous data , 2018 .

[56]  Yu Wang,et al.  Detection and classification of anomaly intrusion using hierarchy clustering and SVM , 2016, Secur. Commun. Networks.

[57]  Kyriakos Stefanidis,et al.  An HMM-Based Anomaly Detection Approach for SCADA Systems , 2016, WISTP.

[58]  M. A. Jabbar,et al.  A Novel Intrusion Detection System Using Artificial Neural Networks and Feature Subset Selection , 2018, International Journal of Engineering & Technology.

[59]  Neminath Hubballi,et al.  OCPAD: One class Naive Bayes classifier for payload based anomaly detection , 2016, Expert Syst. Appl..

[60]  André Zúquete,et al.  Traffic classification and verification using unsupervised learning of Gaussian Mixture Models , 2015, 2015 IEEE International Workshop on Measurements & Networking (M&N).

[61]  Dhruba K. Bhattacharyya,et al.  Network Anomaly Detection: A Machine Learning Perspective , 2013 .

[62]  Wei Chen,et al.  A survey of network anomaly visualization , 2017, Science China Information Sciences.

[63]  Luca Benini,et al.  A semisupervised autoencoder-based approach for anomaly detection in high performance computing systems , 2019, Eng. Appl. Artif. Intell..

[64]  Mourad Khayati,et al.  2015 Ieee International Conference on Big Data (big Data) Online Anomaly Detection over Big Data Streams , 2022 .