Evading Machine Learning Botnet Detection Models via Deep Reinforcement Learning

Botnets are one of predominant threats to Internet security. To date, machine learning technology has wide application in botnet detection because that it is able to summarize the features of existing attacks and generalize to never-before-seen botnet families. However, recent works in adversarial machine learning have shown that attackers are able to bypass the detection model by constructing specific samples, which due to many algorithms are vulnerable to almost imperceptible perturbations of their inputs. According to the degree of adversaries' knowledge about the model, adversarial attacks can be classified into several groups, such as gradient- and score-based attacks. In this paper, we propose a more general framework based on deep reinforcement learning (DRL), which effectively generates adversarial traffic flows to deceive the detection model by automatically adding perturbations to samples. Throughout the process, the target detector will be regarded as a black box and more close to realistic attack circumstance. A reinforcement learning agent is equipped for updating the adversarial samples by combining the feedback from the target model (i.e. benign or malicious) and the sequence of actions, which is able to change the temporal and spatial features of the traffic flows while maintaining the original functionality and executability. The experiment results show that the evasion rates of adversarial botnet flows are significantly improved. Furthermore, with the perspective of defense, this research can help the detection model spot its defect and thus enhance the robustness.

[1]  Christopher Kruegel Proceedings of the 4th USENIX conference on Large-scale exploits and emergent threats , 2011 .

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Ying Tan,et al.  Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN , 2017, DMBD.

[4]  A. Nur Zincir-Heywood,et al.  How to choose from different botnet detection systems? , 2016, NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium.

[5]  J. Doug Tygar,et al.  Adversarial machine learning , 2019, AISec '11.

[6]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.

[7]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[8]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[9]  Patrick D. McDaniel,et al.  Adversarial Perturbations Against Deep Neural Networks for Malware Classification , 2016, ArXiv.

[10]  Pablo Torres,et al.  An analysis of Recurrent Neural Networks for Botnet detection behavior , 2016, 2016 IEEE Biennial Congress of Argentina (ARGENCON).

[11]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[12]  Radu State,et al.  BotTrack: Tracking Botnets Using NetFlow and PageRank , 2011, Networking.

[13]  Matthias Bethge,et al.  Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models , 2017, ICLR.

[14]  Hyrum S. Anderson,et al.  DeepDGA: Adversarially-Tuned Domain Generation and Detection , 2016, AISec@CCS.

[15]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[16]  Jong-Hoon Oh,et al.  Neural networks : the statistical mechanics perspective : proceedings of the CTP-PBSRI Joint Workshop on Theoretical Physics, POSTECH, Pohang, Korea, 2-4 February 95 , 1995 .

[17]  Harris Drucker,et al.  Learning algorithms for classification: A comparison on handwritten digit recognition , 1995 .

[18]  Ali A. Ghorbani,et al.  Botnet detection based on traffic behavior analysis and flow intervals , 2013, Comput. Secur..

[19]  Ming Zhu,et al.  Malware traffic classification using convolutional neural network for representation learning , 2017, 2017 International Conference on Information Networking (ICOIN).

[20]  Sung-Ju Lee,et al.  Systematic Mining of Associated Server Herds for Malware Campaign Discovery , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[21]  Leyla Bilge,et al.  Disclosure: detecting botnet command and control servers through large-scale NetFlow analysis , 2012, ACSAC '12.