Learning Meanings of Words and Constructions, Grounded in a Virtual Game

We discuss the use of data from a virtual world game for automated learning of words and grammatical constructions and their meanings. The language data is an integral part of the social interaction in the game and consists of chat dialogue, which is only constrained by the cultural context, as set by the nature of the provided virtual environment. This paper presents a preliminary exploration of syntactic and semantic aspects of the dialogue in the corpus. We show how simple association metrics can be used to extract words, phrases and more abstract syntactic patterns with targeted meanings or speech-act functions, by making use of the nonlinguistic context.

[1]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[2]  R. Stainton Perspectives in the Philosophy of Language: A Concise Anthology , 2000 .

[3]  W. Bruce Croft Radical Construction Grammar , 2001 .

[4]  L. Steels Evolving grounded communication for robots , 2003, Trends in Cognitive Sciences.

[5]  Ipke Wachsmuth,et al.  Max - A Multimodal Assistant in Virtual Reality Construction , 2003, Künstliche Intell..

[6]  Stefan Th. Gries,et al.  Collostructions: Investigating the interaction of words and constructions , 2003 .

[7]  M. Tomasello,et al.  Early syntactic creativity: a usage-based approach. , 2003, Journal of child language.

[8]  Deb Roy,et al.  Grounded Semantic Composition for Visual Scenes , 2011, J. Artif. Intell. Res..

[9]  P. Kantor Foundations of Statistical Natural Language Processing , 2001, Information Retrieval.

[10]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[11]  Deb Roy,et al.  Probabilistic grounding of situated speech using plan recognition and reference resolution , 2005, ICMI '05.

[12]  Deb Roy,et al.  Semiotic schemas: A framework for grounding language in action and perception , 2005, Artif. Intell..

[13]  Michael Fleischman,et al.  Why Verbs are Harder to Learn than Nouns: Initial Insights from a Computational Model of Intention Recognition in Situated Word Learning , 2005 .

[14]  Paul R. Cohen,et al.  Wubble World , 2007, AIIDE.

[15]  Jeff Orkin,et al.  Learning plan networks in conversational video games , 2007 .

[16]  Jimmy J. Lin,et al.  Overview of the TREC 2007 Question Answering Track , 2008, TREC.

[17]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[18]  Jeff Orkin,et al.  The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online , 2008, J. Game Dev..

[19]  Jeff Orkin,et al.  Automatic learning and generation of social behavior from collective human gameplay , 2009, AAMAS.