Following directions using statistical machine translation

Mobile robots that interact with humans in an intuitive way must be able to follow directions provided by humans in unconstrained natural language. In this work we investigate how statistical machine translation techniques can be used to bridge the gap between natural language route instructions and a map of an environment built by a robot. Our approach uses training data to learn to translate from natural language instructions to an automatically-labeled map. The complexity of the translation process is controlled by taking advantage of physical constraints imposed by the map. As a result, our technique can efficiently handle uncertainty in both map labeling and parsing. Our experiments demonstrate the promising capabilities achieved by our approach.

[1]  J. Y. Yen.  Finding the K Shortest Loopless Paths in a Network , 1971 .

[2]  Alfred V. Aho,et al.  The Theory of Parsing, Translation, and Compiling , 1972 .

[3]  John Cocke,et al.  A Statistical Approach to Machine Translation , 1990, CL.

[4]  Ernesto de Queirós Vieira Martins,et al.  A new improvement for a K shortest paths algorithm , 2001 .

[5]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[6]  Deb K. Roy,et al.  Learning visually grounded words and syntax for a scene description task , 2002, Comput. Speech Lang..

[7]  Hermann Ney,et al.  A Systematic Comparison of Various Statistical Alignment Models , 2003, CL.

[8]  Marjorie Skubic,et al.  Spatial language for human-robot dialogs , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Deb Roy,et al.  Semiotic schemas: A framework for grounding language in action and perception , 2005, Artif. Intell..

[10]  J. Andrew Bagnell,et al.  Maximum margin planning , 2006, ICML.

[11]  David Chiang,et al.  An Introduction to Synchronous Grammars , 2006 .

[12]  Raymond J. Mooney,et al.  Learning for Semantic Parsing with Statistical Machine Translation , 2006, NAACL.

[13]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.

[14]  Raymond J. Mooney,et al.  Learning for semantic parsing and natural language generation using statistical machine translation techniques , 2007 .

[15]  Dieter Fox,et al.  Voronoi Random Fields: Extracting Topological Structure of Indoor Environments via Place Labeling , 2007, IJCAI.

[16]  Raymond J. Mooney,et al.  Learning to sportscast: a test of grounded language acquisition , 2008, ICML '08.

[17]  Raymond J. Mooney,et al.  Learning to Connect Language and Perception , 2008, AAAI.

[18]  Anind K. Dey,et al.  Navigate like a cabbie: probabilistic reasoning from observed context-aware behavior , 2008, UbiComp.

[19]  Masahiro Fujita,et al.  3D Perception and Environment Map Generation for Humanoid Robot Navigation , 2008, Int. J. Robotics Res..

[20]  Stefanie Tellex,et al.  Object schemas for grounding language in a responsive robot , 2008, Connect. Sci..

[21]  Adam Lopez,et al.  Statistical machine translation , 2008, AMTA.

[22]  A. Haas,et al.  Learning to Follow Navigational Route Instructions , 2009, IJCAI.

[23]  Tadayoshi Kohno,et al.  A spotlight on security and privacy risks with future household robots: attacks and lessons , 2009, UbiComp.

[24]  Matthias Scheutz,et al.  What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution , 2009, 2009 IEEE International Conference on Robotics and Automation.

[25]  Luke S. Zettlemoyer,et al.  Reinforcement Learning for Mapping Instructions to Actions , 2009, ACL.

[26]  Nicholas Roy,et al.  Where to go: Interpreting natural directions using global inference , 2009, 2009 IEEE International Conference on Robotics and Automation.