Player Goal Recognition in Open-World Digital Games with Long Short-Term Memory Networks

Recent years have seen a growing interest in player modeling for digital games. Goal recognition, which aims to accurately recognize players' goals from observations of low-level player actions, is a key problem in player modeling. However, player goal recognition poses significant challenges because of the inherent complexity and uncertainty pervading gameplay. In this paper, we formulate player goal recognition as a sequence labeling task and introduce a goal recognition framework based on long short-term memory (LSTM) networks. Results show that LSTM-based goal recognition is significantly more accurate than previous state-of-the-art methods, including n-gram encoded feedforward neural networks pre-trained with stacked denoising autoencoders, as well as Markov logic network-based models. Because of increased goal recognition accuracy and the elimination of labor-intensive feature engineering, LSTM-based goal recognition provides an effective solution to a central problem in player modeling for open-world digital games.

[1]  Henry A. Kautz,et al.  Generalized Plan Recognition , 1986, AAAI.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[4]  A. Piperno,et al.  2003 , 2003, Intensive Care Medicine.

[5]  James C. Lester,et al.  Probabilistic Goal Recognition in Interactive Narrative Environments , 2006, AAAI.

[6]  K. Squire.  Open-Ended Video Games: A Model for Developing Learning for the Interactive Age , 2007 .

[7]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[8]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[9]  Svetha Venkatesh,et al.  Efficient duration and hierarchical modeling for human activity recognition , 2009, Artif. Intell..

[10]  Robert P. Goldman,et al.  A probabilistic plan recognition algorithm based on plan tree grammars , 2009, Artif. Intell..

[11]  Christopher D. Manning,et al.  Learning Continuous Phrase Representations and Syntactic Parsing with Recursive Neural Networks , 2010 .

[12]  Raymond J. Mooney,et al.  Abductive Markov Logic for Plan Recognition , 2011, Proceedings of the AAAI Conference on Artificial Intelligence.

[13]  Jonathan P. Rowe,et al.  Integrating Learning, Problem Solving, and Engagement in Narrative-Centered Learning Environments , 2011, Int. J. Artif. Intell. Educ..

[14]  Rafael Bidarra,et al.  In Press: Ieee Transactions on Computational Intelligence and Ai in Games Adaptivity Challenges in Games and Simulations: a Survey , 2022 .

[15]  Jonathan P. Rowe,et al.  Goal Recognition with Markov Logic Networks for Player-Adaptive Games , 2011, AIIDE.

[16]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[17]  H. Seiden.  Plan , 2012 .

[18]  Julian Togelius,et al.  Artificial and Computational Intelligence in Games , 2013, Artificial and Computational Intelligence in Games.

[19]  Froduald Kabanza,et al.  Controlling the Hypothesis Space in Probabilistic Plan Recognition , 2013, IJCAI.

[20]  Vadim Bulitko,et al.  Interactive Narrative: An Intelligent Systems Approach , 2012, AI Mag..

[21]  Daniele Loiacono,et al.  Player Modeling , 2013, Artificial and Computational Intelligence in Games.

[22]  Robert P. Goldman,et al.  Plan, Activity, and Intent Recognition: Theory and Practice , 2014 .

[23]  Jonathan P. Rowe,et al.  Generalizability of Goal Recognition Models in Narrative-Centered Learning Environments , 2014, UMAP.

[24]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[25]  Jonathan P. Rowe,et al.  Deep Learning-Based Goal Recognition in Open-Ended Digital Games , 2014, AIIDE.

[26]  Hugo Larochelle,et al.  Using a Recursive Neural Network to Learn an Agent's Decision Model for Plan Recognition , 2015, IJCAI.

[27]  Stephen G. Ware,et al.  A Survey and Analysis of Techniques for Player Behavior Prediction in Massively Multiplayer Online Role-Playing Games , 2015, IEEE Transactions on Emerging Topics in Computing.

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..