Reading Between the Lines : Learning to Map High-level Instructions to

In this paper, we address the task of mapping high-level instructions to sequences of commands in an external environment. Processing these instructions is challenging—they posit goals to be achieved without specifying the steps required to complete them. We describe a method that fills in missing information using an automatically derived environment model that encodes states, transitions, and commands that cause these transitions to happen. We present an efficient approximate approach for learning this environment model as part of a policygradient reinforcement learning algorithm for text interpretation. This design enables learning for mapping high-level instructions, which previous statistical methods cannot handle.1

[1]  Terry Winograd,et al.  Understanding natural language , 1974 .

[2]  John E. Moody,et al.  Note on Learning Rate Schedules for Stochastic Optimization , 1990, NIPS.

[3]  Barbara Di Eugenio,et al.  Understanding Natural Language Instructions: The Case of Purpose Clauses , 1992, ACL.

[4]  Barbara Di Eugenio,et al.  On the Interpretation of Natural Language Instructions , 1992, COLING.

[5]  Andrew W. Moore,et al.  Generalization in Reinforcement Learning: Safely Approximating the Value Function , 1994, NIPS.

[6]  Norman I. Badler,et al.  Instructions, Intentions and Expectations , 1995, Artif. Intell..

[7]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[8]  Paul R. Cohen,et al.  Grounding knowledge in sensors: unsupervised learning for language and planning , 2001 .

[9]  Jeffrey Mark Siskind,et al.  Grounding the Lexical Semantics of Verbs in Visual Perception using Force Dynamics and Event Logic , 1999, J. Artif. Intell. Res..

[10]  S. Singh,et al.  Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System , 2011, J. Artif. Intell. Res..

[11]  Alex Lascarides,et al.  Imperatives in dialogue , 2003 .

[12]  Chen Yu,et al.  On the Integration of Grounding Language and Learning Objects , 2004, AAAI.

[13]  Deb Roy,et al.  Intentional Context in Situated Natural Language Learning , 2005, CoNLL.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.

[16]  Peter Stone,et al.  Model-based function approximation in reinforcement learning , 2007, AAMAS '07.

[17]  Raymond J. Mooney,et al.  Learning to Connect Language and Perception , 2008, AAAI.

[18]  Dan Roth,et al.  Reading to Learn: Constructing Features from Semantic Abstracts , 2009, EMNLP.

[19]  Oliver Lemon,et al.  User Simulations for Context-Sensitive Speech Recognition in Spoken Dialogue Systems , 2009, EACL.

[20]  Steve J. Young,et al.  The Hidden Agenda User Simulation Model , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  Dan Klein,et al.  Learning Semantic Correspondences with Less Supervision , 2009, ACL.

[22]  Luke S. Zettlemoyer,et al.  Reinforcement Learning for Mapping Instructions to Actions , 2009, ACL.

[23]  Micah Z. Brodsky,et al.  WikiDo , 2009, HotNets.

[24]  Dieter Fox,et al.  Following directions using statistical machine translation , 2010, HRI 2010.