Unsupervised Text Classification for Natural Language Interactive Narratives

Natural language interactive narratives are a variant of traditional branching storylines where player actions are expressed in natural language rather than by selecting among choices. Previous efforts have handled the richness of natural language input using machine learning technologies for text classification, bootstrapping supervised machine learning approaches with human-in-the-loop data acquisition or by using expected player input as fake training data. This paper explores a third alternative, where unsupervised text classifiers are used to automatically route player input to the most appropriate storyline branch. We describe the Data-driven Interactive Narrative Engine (DINE), a web-based tool for authoring and deploying natural language interactive narratives. To compare the performance of different algorithms for unsupervised text classification, we collected thousands of user inputs from hundreds of crowdsourced participants playing 25 different scenarios, and hand-annotated them to create a goldstandard test set. Through comparative evaluations, we identified an unsupervised algorithm for narrative text classification that approaches the performance of supervised text classification algorithms. We discuss how this technology supports authors in the rapid creation and deployment of interactive narrative experiences, with authorial burdens similar to that of traditional branching storylines.

[1]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[2]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[3]  Arnav Jhala,et al.  Branching Storylines in Virtual Reality Environments for Leadership Development , 2004, AAAI.

[4]  R. Swanson,et al.  Identifying Personal Stories in Millions of Weblog Entries , 2009, ICWSM 2009.

[5]  Andrew Stern,et al.  Integrating Plot, Character and Natural Language Processing in the Interactive Drama Façade , 2003 .

[6]  Kallirroi Georgila,et al.  New Dimensions in Testimony: Digitally Preserving a Holocaust Survivor's Interactive Storytelling , 2015, ICIDS.

[7]  Reid Swanson,et al.  Say Anything: Using Textual Case-Based Reasoning to Enable Open-Domain Interactive Storytelling , 2012, TIIS.

[8]  Zornitsa Kozareva,et al.  SemEval-2012 Task 7: Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning , 2011, *SEMEVAL.

[9]  Andrew S. Gordon,et al.  Creative Help: A Story Writing Assistant , 2015, ICIDS.

[10]  Randall W. Hill,et al.  Toward a New Generation of Virtual Humans for Interactive Experiences , 2002, IEEE Intell. Syst..

[11]  Nick Montfort,et al.  Twisty Little Passages: An Approach to Interactive Fiction , 2003 .

[12]  David R. Traum,et al.  SAWDUST: a Semi-Automated Wizard Dialogue Utterance Selection Tool for domain-independent large-domain dialogue , 2014, SIGDIAL Conference.

[13]  Cosmin Adrian Bejan,et al.  Commonsense Causal Reasoning Using Millions of Personal Stories , 2011, AAAI.

[14]  David R. Traum,et al.  Improving question-answering with linking dialogues , 2006, IUI '06.

[15]  Randall W. Hill,et al.  Guided Conversations about Leadership: Mentoring with Movies and Interactive Characters , 2003, IAAI.

[16]  Edward Packard,et al.  The Cave of Time , 1979 .

[17]  J. Treble,et al.  Computer Intensive Methods for Hypothesis Testing , 1990 .