Network Traffic Anomaly Detection via Deep Learning

Network intrusion detection is a key pillar towards the sustainability and normal operation of information systems. Complex threat patterns and malicious actors are able to cause severe damages to cyber-systems. In this work, we propose novel Deep Learning formulations for detecting threats and alerts on network logs that were acquired by pfSense, an open-source software that acts as firewall on FreeBSD operating system. pfSense integrates several powerful security services such as firewall, URL filtering, and virtual private networking among others. The main goal of this study is to analyse the logs that were acquired by a local installation of pfSense software, in order to provide a powerful and efficient solution that controls traffic flow based on patterns that are automatically learnt via the proposed, challenging DL architectures. For this purpose, we exploit the Convolutional Neural Networks (CNNs), and the Long Short Term Memory Networks (LSTMs) in order to construct robust multi-class classifiers, able to assign each new network log instance that reaches our system into its corresponding category. The performance of our scheme is evaluated by conducting several quantitative experiments, and by comparing to state-of-the-art formulations.

[1]  M. A. Jabbar,et al.  Random Forest Modeling for Network Intrusion Detection System , 2016 .

[2]  Tram Truong-Huu,et al.  An Empirical Study on Unsupervised Network Anomaly Detection using Generative Adversarial Networks , 2020, Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial Intelligence.

[3]  Amutha Prabakar Muniyandi,et al.  Network Anomaly Detection by Cascading K-Means Clustering and C4.5 Decision Tree algorithm , 2012 .

[4]  Peter Steenkiste,et al.  Network Anomaly Detection Using Co-clustering , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[5]  Georg Langs,et al.  f‐AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks , 2019, Medical Image Anal..

[6]  R. Sekar,et al.  A high-performance network intrusion detection system , 1999, CCS '99.

[7]  Wei Xu,et al.  CNN-RNN: A Unified Framework for Multi-label Image Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Vinod Kumar,et al.  Signature Based Intrusion Detection System Using SNORT , 2012 .

[9]  Bihua Tang,et al.  A Semi-Supervised Learning Approach to IEEE 802.11 Network Anomaly Detection , 2019, 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring).

[10]  Muhammad Munwar Iqbal,et al.  Enhanced Network Anomaly Detection Based on Deep Neural Networks , 2018, IEEE Access.

[11]  Vijay Varadharajan,et al.  A Detailed Investigation and Analysis of Using Machine Learning Techniques for Intrusion Detection , 2019, IEEE Communications Surveys & Tutorials.

[12]  Artemis Voulkidis,et al.  Incidents Information Sharing Platform for Distributed Attack Detection , 2020, IEEE Open Journal of the Communications Society.

[13]  Christopher Krügel,et al.  Using Decision Trees to Improve Signature-Based Intrusion Detection , 2003, RAID.

[14]  Kaushik Roy,et al.  Using a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) to Classify Network Attacks , 2020, Inf..

[15]  Jinoh Kim,et al.  A survey of deep learning-based network anomaly detection , 2017, Cluster Computing.

[16]  Md. Rezaul Karim,et al.  A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network , 2019, Symmetry.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Konstantin Eckle,et al.  A comparison of deep networks with ReLU activation function and linear spline-type methods , 2018, Neural Networks.

[19]  Dong Seong Kim,et al.  Genetic algorithm to improve SVM based network intrusion detection system , 2005, 19th International Conference on Advanced Information Networking and Applications (AINA'05) Volume 1 (AINA papers).

[20]  Alexander Binder,et al.  Deep Semi-Supervised Anomaly Detection , 2019, ICLR.

[21]  J. I N Y U,et al.  Choosing the content of textual summaries of large time-series data sets , 2005 .

[22]  Artemis Voulkidis,et al.  Proactive Critical Energy Infrastructure Protection via Deep Feature Learning , 2020, Energies.

[23]  Matthieu Roy,et al.  Experience Report: Log Mining Using Natural Language Processing and Application to Anomaly Detection , 2017, 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE).

[24]  Naveen K. Chilamkurti,et al.  Survey on SDN based network intrusion detection system using machine learning approaches , 2018, Peer-to-Peer Networking and Applications.

[25]  Yu-Chiang Frank Wang,et al.  Learning Deep Latent Spaces for Multi-Label Classification , 2017, ArXiv.

[26]  Zeynab Abbasi Khalifelu,et al.  Analysis and evaluation of unstructured data: text mining versus natural language processing , 2011, 2011 5th International Conference on Application of Information and Communication Technologies (AICT).

[27]  Manuel López Martín,et al.  Network intrusion detection with a novel hierarchy of distances between embeddings of hash IP addresses , 2021, Knowl. Based Syst..

[28]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[29]  Bo Zong,et al.  A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data , 2018, AAAI.

[30]  Priyanka Sharma,et al.  A Review paper on pfsense – an Open source firewall introducing with different capabilities & customization , 2017 .

[31]  D Vasumathi,et al.  Review on anomaly based network intrusion detection system , 2017, 2017 International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT).

[32]  Vassilios G. Vassilakis,et al.  Denial-of-Service Attacks and Countermeasures in the RPL-Based Internet of Things , 2019, CyberICPS/SECPRE/SPOSE/ADIoT@ESORICS.

[33]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[34]  Marius Kloft,et al.  Toward Supervised Anomaly Detection , 2014, J. Artif. Intell. Res..

[35]  Subutai Ahmad,et al.  Unsupervised real-time anomaly detection for streaming data , 2017, Neurocomputing.

[36]  Yanchun Zhang,et al.  Supervised Anomaly Detection in Uncertain Pseudoperiodic Data Streams , 2016, ACM Trans. Internet Techn..

[37]  A. K. Rigler,et al.  Accelerating the convergence of the back-propagation method , 1988, Biological Cybernetics.

[38]  Raj Kamal,et al.  An improved random forest classifier for multi-class classification , 2016 .

[39]  Francesco Cricri,et al.  Clustering and Unsupervised Anomaly Detection with l2 Normalized Deep Auto-Encoder Representations , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[40]  John W. Lockwood,et al.  Fast and Scalable Pattern Matching for Network Intrusion Detection Systems , 2006, IEEE Journal on Selected Areas in Communications.

[41]  Aidong Men,et al.  A Hybrid Semi-Supervised Anomaly Detection Model for High-Dimensional Data , 2017, Comput. Intell. Neurosci..

[42]  Shengcai Liao,et al.  Soft-Margin Softmax for Deep Classification , 2017, ICONIP.

[43]  Neeraj Kumar,et al.  Machine Learning Models for Secure Data Analytics: A taxonomy and threat model , 2020, Comput. Commun..

[44]  Lifeng Cao,et al.  Analysis of Multi-Types of Flow Features Based on Hybrid Neural Network for Improving Network Anomaly Detection , 2019, IEEE Access.

[45]  Jinoh Kim,et al.  An Empirical Study on Network Anomaly Detection Using Convolutional Neural Networks , 2018, 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS).

[46]  Keiron O'Shea,et al.  An Introduction to Convolutional Neural Networks , 2015, ArXiv.

[47]  Ioannis D. Moscholios,et al.  A Signature-based Intrusion Detection System for the Internet of Things , 2018 .

[48]  M. N. Sulaiman,et al.  A Review On Evaluation Metrics For Data Classification Evaluations , 2015 .

[49]  Suleyman Serdar Kozat,et al.  Unsupervised Anomaly Detection With LSTM Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Toby P. Breckon,et al.  GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training , 2018, ACCV.

[51]  Yang Wang,et al.  Boosting for Learning Multiple Classes with Imbalanced Class Distribution , 2006, Sixth International Conference on Data Mining (ICDM'06).

[52]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[53]  Jason Weston,et al.  Deep learning via semi-supervised embedding , 2008, ICML '08.

[54]  Hiroshi Takahashi,et al.  Autoencoding Binary Classifiers for Supervised Anomaly Detection , 2019, PRICAI.

[55]  Lalu Banoth,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2017 .

[56]  E. Valuations,et al.  A R EVIEW ON E VALUATION M ETRICS F OR D ATA C LASSIFICATION E VALUATIONS , 2015 .

[57]  Svetha Venkatesh,et al.  Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[58]  Hamid H. Jebur,et al.  Machine Learning Techniques for Anomaly Detection: An Overview , 2013 .

[59]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.