Interpretation of Spatial Language in a Map Navigation Task

We have developed components of an automated system that understands and follows navigational instructions. The system has prior knowledge of the geometry and landmarks of specific maps. This knowledge is exploited to infer complex paths through maps based on natural language descriptions. The approach is based on an analysis of verbal commands in terms of elementary semantic units that are composed to generate a probability distribution over possible spatial paths in a map. An integration mechanism based on dynamic programming guides this language-to-path translation process, ensuring that resulting paths satisfy continuity and smoothness criteria. In the current implementation, parsing of text into semantic units is performed manually. Composition and interpretation of semantic units into spatial paths is performed automatically. In the evaluations, we show that the system accurately predicts the speakers' intended meanings for a range of instructions. This paper provides building blocks for a complete system that, when combined with robust parsing technologies, could lead to a fully automatic spatial language interpretation system

[1]  Hubert D. Zimmer,et al.  Critical Features for the Selection of Verbal Descriptions for Path Relations , 2001 .

[2]  Andrew U. Frank,et al.  Theories and Methods of Spatio-Temporal Reasoning in Geographic Space , 1992, Lecture Notes in Computer Science.

[3]  Christopher Riesbeck,et al.  "You Can't Miss it!": Judging the Clarity of Directions , 1980, Cogn. Sci..

[4]  Richard M. Schwartz,et al.  An Algorithm that Learns What's in a Name , 1999, Machine Learning.

[5]  R. Power.  The organisation of purposeful dialogues , 1979 .

[6]  Max J. Egenhofer,et al.  Metric details for natural-language spatial relations , 1998, TOIS.

[7]  Deb Roy,et al.  Semiotic schemas: A framework for grounding language in action and perception , 2005, Artif. Intell..

[8]  Anne H. Anderson,et al.  The Hcrc Map Task Corpus , 1991 .

[9]  Guido Bugmann,et al.  Corpus-Based Robotics: A Route Instruction Example , 2003 .

[10]  Deb Roy,et al.  Grounding words in perception and action : computational insights , 2005 .

[11]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[12]  Kerstin Fischer,et al.  Spatial Strategies in Human-Robot Communication , 2002, Künstliche Intell..

[13]  Gordon D. Logan,et al.  A computational analysis of the apprehension of spatial relations , 1996 .

[14]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[15]  Leonard Talmy,et al.  How Language Structures Space , 1983 .

[16]  S. Levinson.  Frames of reference and Molyneux's question: Cross-linguistic evidence , 1996 .

[17]  M. Denis.  The description of routes : A cognitive approach to the production of spatial discourse , 1997 .

[18]  Reid G. Simmons,et al.  GRACE: An Autonomous Robot for the AAAI Robot Challenge , 2003, AI Mag..

[19]  Laura A. Carlson,et al.  Grounding spatial language in perception: an empirical and computational investigation. , 2001, Journal of experimental psychology. General.

[20]  Stefanie Tellex,et al.  Spatial routines for a simulated speech-controlled vehicle , 2006, HRI '06.

[21]  Rick Grush,et al.  Self, World and Space: The Meaning and Mechanismsof Ego- and Allocentric Spatial Representation , 2000 .

[22]  Daniel Hernández,et al.  Qualitative Representation of Spatial Knowledge , 1994, Lecture Notes in Computer Science.

[23]  David Chapman,et al.  What are plans for? , 1990, Robotics Auton. Syst..

[24]  Paul U. Lee,et al.  Pictorial and Verbal Tools for Conveying Routes , 1999, COSIT.

[25]  Klaus-Peter Gapp.  Angle, Distance, Shape, and their Relationship to Projective Relations , 1995 .

[26]  大野 義夫,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[27]  Oscar Mayora-Ibarra,et al.  Robotic Remote Navigation by Speech Commands with Automatic Obstacles Detection , 2003, Robotics and Applications.

[28]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[29]  Daniel R. Montello,et al.  The Geometry of Environmental Knowledge , 1992, Spatio-Temporal Reasoning.

[30]  D. Roy.  Grounding words in perception and action: computational insights , 2005, Trends in Cognitive Sciences.