Customizing Scripted Bots: Sample Efficient Imitation Learning for Human-like Behavior in Minecraft

Although there have been many advances in machine learning and artificial intelligence, video games still define the behavior of characters through careful scripting. These scripted agents follow a fixed set of rules or behavior trees resulting in behavior which is predictable and discernibly different from human gameplay. We demonstrate how to combine imitation learning with scripted agents in order to efficiently train hierarchical policies. Integrating both programmed and learned components, these hierarchical polices improve upon the expressiveness of the original scripted agent, allowing more diverse and human-like behavior to emerge. Additionally, we demonstrate that learned policies also maintain behavior guarantees afforded by the scripted agent, allowing a developer to be confident in the behavior of the learned policy. We demonstrate this interplay between classical AI techniques and statistical machine learning through a case study in Minecraft, and explore open questions about the necessary and sufficient conditions to achieve a fruitful interplay between these two approaches.