Threat, Explore, Barter, Puzzle: A Semantically-Informed Algorithm for Extracting Interaction Modes

In the world of online gaming, not all actions are created equal. For example, when a player’s character is confronted with a closed door, it would not make much sense to brandish a weapon, apply a healing potion, or attempt to barter. A more reasonable response would be to either open or unlock the door. The term interaction mode embodies the idea that many potential actions are neither useful nor applicable in a given situation. This paper presents a AEGIM, an algorithm for the automated extraction of game interaction modes via a semantic embedding space. AEGIM uses an image captioning system in conjunction with a semantic vector space model to create a gestalt representation of in-game screenshots, thus enabling it to detect the interaction mode evoked by the game.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  Michael Mateas,et al.  Automated game design learning , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[3]  Thomas Demeester,et al.  Representation learning for very short texts using weighted word embedding aggregation , 2016, Pattern Recognit. Lett..

[4]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.

[5]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[6]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[7]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[8]  Chris Sauer,et al.  Beating Atari with Natural Language Guided Reinforcement Learning , 2017, ArXiv.

[9]  三嶋 博之.  The theory of affordances , 2008 .

[10]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[11]  Li Fei-Fei,et al.  Reasoning about Object Affordances in a Knowledge Base Representation , 2014, ECCV.

[12]  Thomas L. Griffiths,et al.  Evaluating Vector-Space Models of Word Representation, or, The Unreasonable Effectiveness of Counting Words Near Other Words , 2017, CogSci.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Cícero Nogueira dos Santos,et al.  Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts , 2014, COLING.

[15]  Jian Sun,et al.  Rich Image Captioning in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  David Wingate,et al.  What Can You Do with a Rock? Affordance Extraction via Word Embeddings , 2017, IJCAI.

[17]  Tom Schaul,et al.  StarCraft II: A New Challenge for Reinforcement Learning , 2017, ArXiv.

[18]  Shafiq R. Joty,et al.  Dis-S2V: Discourse Informed Sen2Vec , 2016, ArXiv.

[19]  Heinz Wörn,et al.  Haptic object recognition for multi-fingered robot hands , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[20]  John E. Laird,et al.  Human-Level AI's Killer Application: Interactive Computer Games , 2000, AI Mag..

[21]  Trevor Darrell,et al.  Learning to Detect Visual Grasp Affordance , 2016, IEEE Transactions on Automation Science and Engineering.