Learning to Parse Natural Language to a Robot Execution System

The ability to interpret, or ground, natural language commands so they can be executed by a robot is important for enabling untrained users to interact with robots. Logic-based formal representations have been applied very successfully to robots’ world and action models taking perceptual and grounding uncertainty into account; however, these approaches have historically either used formal target representations with severely limited expressiveness or written the formal representations manually rather than learning them. As a consequence, they are not able to parse previously unseen NL commands to more complex robot control systems. In this paper, we investigate how recently developed parser learning techniques can be applied to mapping natural language commands to a formal representation expressive enough for modeling robot control systems. Specifically, we aim to learn a parser mapping natural language indoor route instructions into a LISP-like control language. We test our approach using a simulator executing RCL programs on sets of natural language route instructions given by non-experts. 1 Motivation, Problem Statement and Related Work The ability to interpret, or ground, natural language (NL) commands so they can be executed by a robot is important for enabling untrained users to interact with robots. The problem of grounding NL commands encompasses two key components: First, parsing natural language into a formal representation capable of representing a robot and its operation in an environment; and, second, mapping the formal representation to actions and perceptions in the real world. Logic-based formal representations have been applied very successfully to robot control systems, with manual grounding of commands for navigation tasks [2, 5, 4, 9]. A more flexible approach is to learn grounding relations, rather than inputting them manually. One line of work shows how parsed natural