Grounding Language for Interactive Task Learning

This paper describes how language is grounded by a comprehension system called Lucia within a robotic agent called Rosie that can manipulate objects and navigate indoors. The whole system is built within the Soar cognitive architecture and uses Embodied Construction Grammar (ECG) as a formalism for describing linguistic knowledge. Grounding is performed using knowledge from the grammar itself, from the linguistic context, from the agents perception, and from an ontology of long-term knowledge about object categories and properties and actions the agent can perform. The paper also describes a benchmark corpus of 200 sentences in this domain along with test versions of the world model and ontology and gold-standard meanings for each of the sentences. The benchmark is contained in the supplemental materials.

[1]  Benjamin K. Bergen,et al.  Embodied Construction Grammar , 2013 .

[2]  John E. Laird,et al.  Cognitive Modeling Approaches to Language Comprehension Using Construction Grammar , 2017, AAAI Spring Symposia.

[3]  John E. Laird,et al.  Learning General and Efficient Representations of Novel Games Through Interactive Instruction , 2016 .

[4]  John E. Laird,et al.  Toward Integrating Cognitive Linguistics and Cognitive Language Processing , 2016 .

[5]  Aaron Mininger,et al.  Interactively Learning Strategies for Handling References to Unseen or Unknown Objects , 2016 .

[6]  Ross A. Knepper,et al.  Asking for Help Using Inverse Semantics , 2014, Robotics: Science and Systems.

[7]  Luc Steels,et al.  Language Grounding in Robots , 2012, Springer US.

[8]  Jerome A. Feldman,et al.  Best-fit constructional analysis , 2008 .

[9]  John E. Laird,et al.  The Soar Cognitive Architecture , 2012 .

[10]  John E. Laird,et al.  Cognitive Robotics Using the Soar Cognitive Architecture , 2012, CogRob@AAAI.

[11]  Peter Lindes Ambiguity Resolution in a Cognitive Model of Language Comprehension , 2017 .

[12]  Luke S. Zettlemoyer,et al.  Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions , 2013, TACL.

[13]  Jerome A. Feldman,et al.  Application-Independent and Integration-Friendly Natural Language Understanding , 2016, GCAI.

[14]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[15]  Nick Chater,et al.  The Now-or-Never bottleneck: A fundamental constraint on language , 2015, Behavioral and Brain Sciences.

[16]  John E. Laird,et al.  Acquiring Grounded Representations of Words with Situated Interactive Instruction , 2012 .

[17]  Christopher D. Manning,et al.  Learning Language Games through Interaction , 2016, ACL.

[18]  John E. Laird,et al.  Learning task goals interactively with visual demonstrations , 2016, BICA 2016.

[19]  John E. Laird,et al.  Learning Goal-Oriented Hierarchical Tasks from Situated Interactive Instruction , 2014, AAAI.

[20]  Richard L. Lewis,et al.  Integrating knowledge sources in language comprehension , 1993 .