INSOMNIA: Towards Concept-Drift Robustness in Network Intrusion Detection

Despite decades of research in network traffic analysis and incredible advances in artificial intelligence, network intrusion detection systems based on machine learning (ML) have yet to prove their worth. One core obstacle is the existence of concept drift, an issue for all adversary-facing security systems. Additionally, specific challenges set intrusion detection apart from other ML-based security tasks, such as malware detection. In this work, we offer a new perspective on these challenges. We propose INSOMNIA, a semi-supervised intrusion detector which continuously updates the underlying ML model as network traffic characteristics are affected by concept drift. We use active learning to reduce latency in the model updates, label estimation to reduce labeling overhead, and apply explainable AI to better interpret how the model reacts to the shifting distribution. To evaluate INSOMNIA, we extend TESSERACT - a framework originally proposed for performing sound time-aware evaluations of ML-based malware detectors - to the network intrusion domain. Our evaluation shows that accounting for drifting scenarios is vital for effective intrusion detection systems.

[1]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[2]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[3]  Gianluca Stringhini,et al.  Tiresias: Predicting Security Events Through Deep Learning , 2018, CCS.

[4]  Michele Colajanni,et al.  Analysis of high volumes of network traffic for Advanced Persistent Threat detection , 2016, Comput. Networks.

[5]  Vinod Yegneswaran,et al.  BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation , 2007, USENIX Security Symposium.

[6]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[7]  David D. Cox,et al.  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures , 2013, ICML.

[8]  R. Tibshirani,et al.  Diagnosis of multiple cancer types by shrunken centroids of gene expression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[10]  Marcel van Gerven,et al.  Explainable Deep Learning: A Field Guide for the Uninitiated , 2020, J. Artif. Intell. Res..

[11]  Lorenzo Cavallaro,et al.  TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time , 2018, USENIX Security Symposium.

[12]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.

[13]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[14]  Ali Feizollah,et al.  The Evolution of Android Malware and Android Analysis Techniques , 2017, ACM Comput. Surv..

[15]  Francisco Herrera,et al.  A unifying view on dataset shift in classification , 2012, Pattern Recognit..

[16]  Lalu Banoth,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2017 .

[17]  Cynthia Rudin,et al.  All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously , 2019, J. Mach. Learn. Res..

[18]  P. Biecek,et al.  dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python , 2020, J. Mach. Learn. Res..

[19]  Eduard Ayguadé,et al.  Low-latency multi-threaded ensemble learning for dynamic big data streams , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[20]  K. Rieck,et al.  Dos and Don'ts of Machine Learning in Computer Security , 2020, USENIX Security Symposium.

[21]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[22]  Iqbal Gondal,et al.  Cyberattack triage using incremental clustering for intrusion detection systems , 2019, International Journal of Information Security.

[23]  Marília Curado,et al.  Performance Analysis of Network Traffic Predictors in the Cloud , 2016, Journal of Network and Systems Management.

[24]  Cordelia Schmid,et al.  End-to-End Incremental Learning , 2018, ECCV.

[25]  Naren Ramakrishnan,et al.  Unearthing Stealthy Program Attacks Buried in Extremely Long Execution Paths , 2015, CCS.

[26]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[27]  Feifei Li,et al.  DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning , 2017, CCS.

[28]  Stefan Axelsson,et al.  The base-rate fallacy and the difficulty of intrusion detection , 2000, TSEC.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Corrado Loglisci,et al.  Multi-Channel Deep Feature Learning for Intrusion Detection , 2020, IEEE Access.

[31]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[32]  Corrado Loglisci,et al.  Active learning via collective inference in network regression problems , 2018, Inf. Sci..

[33]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[34]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[35]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[36]  Xinyu Xing,et al.  CADE: Detecting and Explaining Concept Drift Samples for Security Applications , 2021, USENIX Security Symposium.

[37]  Donato Malerba,et al.  GAN augmentation to deal with imbalance in imaging-based intrusion detection , 2021, Future Gener. Comput. Syst..

[38]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[39]  Bidyut Baran Chaudhuri,et al.  A new definition of neighborhood of a point in multi-dimensional space , 1996, Pattern Recognit. Lett..

[40]  Wouter Joosen,et al.  Troubleshooting an Intrusion Detection Dataset: the CICIDS2017 Case Study , 2021, 2021 IEEE Security and Privacy Workshops (SPW).

[41]  Yuval Elovici,et al.  Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection , 2018, NDSS.

[42]  Donato Malerba,et al.  Nearest cluster-based intrusion detection through convolutional neural networks , 2021, Knowl. Based Syst..

[43]  Salvatore J. Stolfo,et al.  Anomalous Payload-Based Worm Detection and Signature Generation , 2005, RAID.

[44]  Ilia Nouretdinov,et al.  Transcend: Detecting Concept Drift in Malware Classification Models , 2017, USENIX Security Symposium.

[45]  Wenke Lee,et al.  Polymorphic Blending Attacks , 2006, USENIX Security Symposium.

[46]  Ali A. Ghorbani,et al.  Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization , 2018, ICISSP.

[47]  Priyadarshini Panda,et al.  Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning , 2018, Neural Networks.

[48]  Guangquan Zhang,et al.  Learning under Concept Drift: A Review , 2019, IEEE Transactions on Knowledge and Data Engineering.

[49]  Lorenzo Cavallaro,et al.  Investigating Labelless Drift Adaptation for Malware Detection , 2021, AISec@CCS.

[50]  Lorenzo Cavallaro,et al.  Enabling Fair ML Evaluations for Security , 2018, CCS.

[51]  Anca Delia Jurcut,et al.  Active Learning for Network Traffic Classification: A Technical Study , 2021, IEEE Transactions on Cognitive Communications and Networking.

[52]  Chunlin Zhang,et al.  Intrusion detection using hierarchical neural networks , 2005, Pattern Recognit. Lett..

[53]  Przemyslaw Biecek,et al.  DALEX: explainers for complex predictive models , 2018, J. Mach. Learn. Res..

[54]  Qi Shi,et al.  A Deep Learning Approach to Network Intrusion Detection , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[55]  Ibrahim F. Tarrad,et al.  Exploiting Incremental Classifiers for the Training of an Adaptive Intrusion Detection Model , 2019, Int. J. Netw. Secur..

[56]  Yinhui Li,et al.  An efficient intrusion detection system based on support vector machines and gradually feature removal method , 2012, Expert Syst. Appl..

[57]  Franco Turini,et al.  A Survey of Methods for Explaining Black Box Models , 2018, ACM Comput. Surv..

[58]  Naveen K. Chilamkurti,et al.  Distributed attack detection scheme using deep learning approach for Internet of Things , 2017, Future Gener. Comput. Syst..

[59]  Ali A. Ghorbani,et al.  Characterization of Tor Traffic using Time based Features , 2017, ICISSP.

[60]  Fakhroddin Noorbehbahani,et al.  An incremental intrusion detection system using a new semi‐supervised stream classification method , 2017, Int. J. Commun. Syst..

[61]  Ali A. Ghorbani,et al.  Characterization of Encrypted and VPN Traffic using Time-related Features , 2016, ICISSP.

[62]  Ke Xu,et al.  DroidEvolver: Self-Evolving Android Malware Detection System , 2019, 2019 IEEE European Symposium on Security and Privacy (EuroS&P).

[63]  Yoshua Bengio,et al.  An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks , 2013, ICLR.

[64]  Philip S. Yu,et al.  Active Learning: A Survey , 2014, Data Classification: Algorithms and Applications.

[65]  Michele Colajanni,et al.  Detection and Threat Prioritization of Pivoting Attacks in Large Networks , 2020, IEEE Transactions on Emerging Topics in Computing.

[66]  Ling Huang,et al.  Reviewer Integration and Performance Measurement for Malware Detection , 2015, DIMVA.

[67]  Marco Loog,et al.  Single Shot Active Learning using Pseudo Annotators , 2018, Pattern Recognit..

[68]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.