CybORG: An Autonomous Cyber Operations Research Gym

Autonomous Cyber Operations (ACO) involves the consideration of blue team (defender) and red team (attacker) decision-making models in adversarial scenarios. To support the application of machine learning algorithms to solve this problem, and to encourage such practitioners to attend to problems in the ACO setting, a suitable gym (toolkit for experiments) is necessary. We introduce CybORG, a work-in-progress gym for ACO research. Driven by the need to efficiently support reinforcement learning to train adversarial decision-making models through simulation and emulation, our design differs from prior related work. Our early evaluation provides some evidence that CybORG is appropriate for our purpose and may provide a basis for advancing ACO research towards practical applications.

[1]  Thomas C. Eskridge,et al.  VINE: A Cyber Emulation Environment for MTD Experimentation , 2015, MTD@CCS.

[2]  Shimon Whiteson,et al.  Protecting against evaluation overfitting in empirical reinforcement learning , 2011, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[3]  Hanna Kurniawati,et al.  Autonomous Penetration Testing using Reinforcement Learning , 2019, ArXiv.

[4]  Shie Mannor,et al.  Reinforcement learning with Gaussian processes , 2005, ICML.

[5]  Mike Hibler,et al.  Large-scale Virtualization in the Emulab Network Testbed , 2008, USENIX ATC.

[6]  Varun Suryan,et al.  Multi-Fidelity Reinforcement Learning with Gaussian Processes , 2017, ArXiv.

[7]  Angelo Furfaro,et al.  A Cloud-based platform for the emulation of complex cybersecurity scenarios , 2018, Future Gener. Comput. Syst..

[8]  Marlos C. Machado,et al.  Generalization and Regularization in DQN , 2018, ArXiv.

[9]  Pieter Abbeel,et al.  Using inaccurate models in reinforcement learning , 2006, ICML.

[10]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[11]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[12]  Ron Alford,et al.  Automated Adversary Emulation : A Case for Planning and Acting with Unknowns , 2018 .

[13]  Carlos Sarraute,et al.  Simulating Cyber-Attacks for Fun and Profit , 2009, SimuTools.

[14]  Daniel R. Tauritz,et al.  Galaxy: A Network Emulation Framework for Cybersecurity , 2018, CSET @ USENIX Security Symposium.

[15]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[16]  Michael Russell,et al.  Validation of network simulation model and scalability tests using example malware , 2016, MILCOM 2016 - 2016 IEEE Military Communications Conference.

[17]  Terry V. Benzel,et al.  The DETER project: Advancing the science of cyber security experimentation and test , 2010, 2010 IEEE International Conference on Technologies for Homeland Security (HST).

[18]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[19]  Doug Miller,et al.  Intelligent, automated red team emulation , 2016, ACSAC.

[20]  Daniel R. Tauritz,et al.  Coevolutionary Agent-based Network Defense Lightweight Event System (CANDLES) , 2015, GECCO.

[21]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[22]  Andreas Krause,et al.  Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Taehoon Kim,et al.  Quantifying Generalization in Reinforcement Learning , 2018, ICML.