Network Intrusion Detection in the Wild - the Orange use case in the SIMARGL project

There is a profuse abundance of network security incidents around the world every day. Increasingly, services and data stored on servers fall victim to sophisticated techniques that cause all sorts of damage. Hackers invent new ways to bypass security measures and modify the existing viruses in order to deceive defense systems. Therefore, in response to these illegal procedures, new ways to defend against them are being developed. In this paper, a method for anomaly detection based on machine learning technique is presented and a near real-time processing system architecture is proposed. The main contribution is a test-run of ML algorithms on real-world data coming from a world-class telecom operator. This work investigates the effectiveness of detecting malicious behaviour in network packets using several machine learning techniques. The results achieved are expressed with a set of metrics. For better clarity on the classifier performance, 10-fold cross-validation was used.

[1]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[2]  Jianbiao Zhang,et al.  CNID: Research of Network Intrusion Detection Based on Convolutional Neural Network , 2020, Discrete Dynamics in Nature and Society.

[3]  Michał Choraś,et al.  A new method of hybrid time window embedding with transformer-based traffic data classification in IoT-networked environment , 2021, Pattern Analysis and Applications.

[4]  Increasing Cell Throughput and Network Capacity in a Real-world HetNet Environment , 2020, 2020 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS).

[5]  H. E. Poston,et al.  A brief taxonomy of intrusion detection strategies , 2012, 2012 IEEE National Aerospace and Electronics Conference (NAECON).

[6]  Devarshi Mehta,et al.  Paper on Searching and Indexing Using Elasticsearch , 2017 .

[7]  Meng Jiang,et al.  Data-Driven Network Intrusion Detection: A Taxonomy of Challenges and Methods , 2020, ArXiv.

[8]  Aiko Pras,et al.  Using NetFlow/IPFIX for Network Management , 2009, Journal of Network and Systems Management.

[9]  Katinka Wolter,et al.  Performance Prediction for the Apache Kafka Messaging System , 2019, 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[10]  Fabio Scotti,et al.  A Comprehensive Survey of Databases and Deep Learning Methods for Cybersecurity and Intrusion Detection Systems , 2020, IEEE Systems Journal.

[11]  Michał Choraś,et al.  A Deep Learning Ensemble for Network Anomaly and Cyber-Attack Detection , 2020, Sensors.

[12]  B. Matthews.  Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[13]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[14]  Zhenpeng Liu,et al.  A Deep Random Forest Model on Spark for Network Intrusion Detection , 2020, Mobile Information Systems.

[15]  Rong Wang,et al.  Real-Time Network Intrusion Detection System Based on Deep Learning , 2019, 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS).

[16]  Sylvain P. Leblanc,et al.  Taxonomy of cyber attacks and simulation of their effects , 2011, SpringSim.

[17]  Bhole Rahul Hiraman,et al.  A Study of Apache Kafka in Big Data Stream Processing , 2018, 2018 International Conference on Information , Communication, Engineering and Technology (ICICET).

[18]  Melnned M. Kantardzic.  Big Data Analytics , 2013, Lecture Notes in Computer Science.

[19]  Rizal Tjut Adek,et al.  A Survey on The Accuracy of Machine Learning Techniques for Intrusion and Anomaly Detection on Public Data Sets , 2020, 2020 International Conference on Data Science, Artificial Intelligence, and Business Analytics (DATABIA).

[20]  C. Essau,et al.  The impact of COVID-19 lockdown on internet use and escapism in adolescents , 2020, Revista de Psicología Clínica con Niños y Adolescentes.

[21]  Katarzyna Wasielewska,et al.  Tight Arms Race: Overview of Current Malware Threats and Trends in Their Detection , 2021, IEEE Access.

[22]  Ashish Kumar Singh,et al.  Distributed Intrusion Detection System using Blockchain and Cloud Computing Infrastructure , 2020, 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI)(48184).

[23]  A. Azzouz.  2011 , 2020, City.

[24]  Michal Choras,et al.  Intrusion detection approach based on optimised artificial neural network , 2020, Neurocomputing.

[25]  D. Gifford.  1975 , 2018, The British Film Catalogue.

[26]  Joshua Zhexue Huang,et al.  Big data analytics on Apache Spark , 2016, International Journal of Data Science and Analytics.

[27]  Xin Li,et al.  LNNLS-KH: A Feature Selection Method for Network Intrusion Detection , 2021, Secur. Commun. Networks.

[28]  Neel Shah,et al.  A framework for social media data analytics using Elasticsearch and Kibana , 2018 .

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.