New And Surprising Ways to Be Mean

Creating Non-Player Characters (NPCs) that can react robustly to unforeseen player behaviour or novel game content is difficult and time-consuming. This hinders the design of believable characters, and the inclusion of NPCs in games that rely heavily on procedural content generation. We have previously addressed this challenge by means of empowerment, a model of intrinsic motivation, and demonstrated how a coupled empowerment maximisation (CEM) policy can yield generic, companion-like behaviour. In this paper, we extend the CEM framework with a minimisation policy to give rise to adversarial behaviour. We conduct a qualitative, exploratory study in a dungeon-crawler game, demonstrating that CEM can exploit the affordances of different content facets in adaptive adversarial behaviour without modifications to the policy. Changes to the level design, underlying mechanics and our character's actions do not threaten our NPC's robustness, but yield new and surprising ways to be mean.

[1]  Henrik Warpefelt,et al.  The Non-Player Character: Exploring the believability of NPC presentation and behavior , 2016 .

[2]  Daan Wierstra,et al.  Variational Intrinsic Control , 2016, ICLR.

[3]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[4]  Patrick van der Smagt,et al.  Unsupervised Real-Time Control Through Variational Empowerment , 2017, ISRR.

[5]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[6]  Pierre-Yves Oudeyer,et al.  How can we define intrinsic motivation , 2008 .

[7]  Christoph Salge,et al.  Empowerment As Replacement for the Three Laws of Robotics , 2017, Front. Robot. AI.

[8]  Kathryn E. Merrick,et al.  Motivated Reinforcement Learning Agents , 2009 .

[9]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[10]  Christoph Salge,et al.  Changing the Environment Based on Empowerment as Intrinsic Motivation , 2014, Entropy.

[11]  Simon Colton,et al.  Supportive and Antagonistic Behaviour in Distributed Computational Creativity via Coupled Empowerment Maximisation , 2016, ICCC.

[12]  Julian Togelius,et al.  Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics , 2018, IEEE Transactions on Games.

[13]  Pierre-Yves Oudeyer,et al.  How can we define intrinsic motivation , 2008 .

[14]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[15]  Gillian Smith,et al.  Understanding procedural content generation: a design-centric analysis of the role of PCG in games , 2014, CHI.

[16]  Chrystopher L. Nehaniv,et al.  Keep Your Options Open: An Information-Based Driving Principle for Sensorimotor Systems , 2008, PloS one.

[17]  Shakir Mohamed,et al.  Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning , 2015, NIPS.

[18]  Kathryn E. Merrick,et al.  Motivated Reinforcement Learning - Curious Characters for Multiuser Games , 2009 .

[19]  W. Marsden.  I and J , 2012 .

[20]  Simon Colton,et al.  Intrinsically motivated general companion NPCs via Coupled Empowerment Maximisation , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).