Spatial Reference in Linguistic Human-Robot Interaction: Iterative, Empirically Supported Development of a Model of Projective Relations

The aim of our research is to enable spontaneous and efficient spatial reference to objects in human-robot interaction situations. This paper presents the iterative, empirically based design of a robotic system that uses a computational model for identifying objects on the basis of a range of spatial reference systems. The efficiency of the system is evaluated by two successive empirical studies involving uninformed users. The linguistic analysis points to the striking variability in speakers' spontaneous strategies and preferences, and it motivates a number of modifications of the computational model.

[1]  F. Bacon.  Where Shall I Go , 1934 .

[2]  H. Grice.  Logic and conversation , 1975 .

[3]  Werner Deutsch,et al.  Psychologie der Objektbenennung , 1976 .

[4]  Annette Herskovits,et al.  Language and spatial cognition , 1986 .

[5]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[6]  Jeremy Peckham,et al.  Recent Developments and Applications of Natural Language Processing , 1989 .

[7]  M. F. Schober.  Spatial perspective-taking in conversation , 1993, Cognition.

[8]  Tim Lüth,et al.  Utilizing Spatial Relations for Natural Language Access to an Autonomous Mobile Agent , 1994, KI.

[9]  Daniel Hernández,et al.  Qualitative Representation of Spatial Knowledge , 1994, Lecture Notes in Computer Science.

[10]  Klaus-Peter Gapp.  Basic Meanings of Spatial Relations: Computation and Evaluation in 3D Space , 1994, AAAI.

[11]  Gerhard Sagerer,et al.  A three-dimensional spatial model for the interpretation of image data , 1998, IJCAI 1995.

[12]  J. Farris.  CONJECTURES AND REFUTATIONS , 1995, Cladistics : the international journal of the Willi Hennig Society.

[13]  Anne Johnstone,et al.  There was a long pause: influencing turn-taking behaviour in human-human and human-computer spoken dialogues , 1995, Int. J. Hum. Comput. Stud..

[14]  Franz Kummert,et al.  Selective Visual Perception Driven by Cues from Speech Processing , 1995 .

[15]  Klaus-Peter Gapp,et al.  Bericht Nr . 135 Processing Spatial Relations in Object Localization Tasks , 1996 .

[16]  Gerhard Sagerer,et al.  Projective Relations for 3D Space: Computational Model, Application, and Psychological Evaluation , 1997, AAAI/IAAI.

[17]  Laura A. Carlson-Radvansky,et al.  The Influence of Reference Frame Selection on Spatial Template Construction , 1997 .

[18]  Jianwei Zhang,et al.  Instructing cooperating assembly robots through situated dialogues in natural language , 1997, Proceedings of International Conference on Robotics and Automation.

[19]  Christian Freksa,et al.  Spatial and Temporal Structures in Cognitive Processes , 1997, Foundations of Computer Science: Potential - Theory - Cognition.

[20]  Ipke Wachsmuth,et al.  Gesture and Sign Language in Human-Computer Interaction , 1997, Lecture Notes in Computer Science.

[21]  H. Rieser.  Repräsentations-Metonymie, Perspektive und Koordination in aufgabenorientierten Dialogen , 1997 .

[22]  M. Carroll,.  Language and conceptualization: Changing place in English and German: language-specific preferences in the conceptualization of spatial relations , 1997 .

[23]  B. MacWhinney.  Models of the emergence of language. , 1998, Annual review of psychology.

[24]  Patrick Oliver,et al.  Representation and Processing of Spatial Expressions , 1998 .

[25]  Jörg Baus,et al.  The Use of Locative Expressions in Dependence of the Spatial Relation Between Target and Reference Object in Two-Dimensional Layouts , 1998, Spatial Cognition.

[26]  K. Kessler.  8 Grounding mental models: Subconceptual dynamics in the resolution of reference in discourse , 1999 .

[27]  Laura A. Carlson,et al.  Selecting a reference frame , 1999, Spatial Cogn. Comput..

[28]  Phillip J. Holcomb,et al.  “Could you hand me those keys on the right?” Disentangling spatial reference frames using different methodologies , 1999, Spatial Cogn. Comput..

[29]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[30]  C. Habel,et al.  Mental Models in Discourse Processing and Reasoning , 1999 .

[31]  Ralf Klabunde.  Logic-Based Choice of Projective Terms , 1999, KI.

[32]  Reinhard Moratz,et al.  Robot Navigation with Schematic Maps , 2000 .

[33]  R. Moratz,et al.  From Communicative Strategies to Cognitive Modelling , 2001 .

[34]  David L. Dowe,et al.  Message Length as an Effective Ockham's Razor in Decision Tree Induction , 2001, International Conference on Artificial Intelligence and Statistics.

[35]  Guido Bugmann,et al.  Using verbal instructions for route learning: Instruction Analysis , 2001 .

[36]  Kerstin Fischer,et al.  Cognitive Modeling of Spatial Reference for Human-Robot Interaction , 2001, Int. J. Artif. Intell. Tools.

[37]  Edward Nelson,et al.  Syntax and Semantics , 1974 .

[38]  E. Zee,et al.  Representing Direction in Language and Space , 2002 .

[39]  Kerstin Fischer,et al.  Spatial Knowledge Representation for Human-Robot Interaction , 2003, Spatial Cognition.

[40]  K. Fischer.  Linguistic Methods for Investigating Concepts in Use , 2003 .

[41]  Philip R. Cohen,et al.  Referring as a Collaborative Process , 2003 .

[42]  R. Moratz,et al.  Instruction modes for joint spatial reference between naive users and a mobile robot , 2003, IEEE International Conference on Robotics, Intelligent Systems and Signal Processing, 2003. Proceedings. 2003.

[43]  Wilfried Brauer,et al.  Spatial Cognition III , 2003, Lecture Notes in Computer Science.

[44]  James M. Keller,et al.  Generating Multi-Level Linguistic Spatial Descriptions from Range Sensor Readings Using the Histogram of Forces , 2003, Auton. Robots.

[45]  S. Levinson.  Space in language and cognition: Explorations in cognitive diversity , 2003 .

[46]  Jianwei Zhang,et al.  A two-arm situated artificial communicator for human-robot cooperative assembly , 2003, IEEE Trans. Ind. Electron..

[47]  Anders Green,et al.  Social and collaborative aspects of interaction with a service robot , 2003, Robotics Auton. Syst..

[48]  Thora Tenbrink,et al.  Identifying Objects on the Basis of Spatial Contrast: An Empirical Study , 2004, Spatial Cognition.

[49]  Anders Green,et al.  Involving users in the design of a mobile office robot , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[50]  Robin R. Murphy,et al.  Moonlight in Miami : A Field Study of Human-Robot Interaction in the Context of an Urban Search and Rescue Disaster Response Training Exercise , 2003 .

[51]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[52]  Robin R. Murphy,et al.  Moonlight in Miami: a field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise , 2004 .

[53]  Jean Scholtz,et al.  Beyond Usability Evaluation: Analysis of Human-Robot Interaction at a Major Robotics Competition , 2004, Hum. Comput. Interact..

[54]  John A. Bateman,et al.  Towards Dialogue Based Shared Control of Navigating Robots , 2004, Spatial Cognition.

[55]  Reinhard Moratz,et al.  Automatic object recognition within an of .ce environment , 2004, First Canadian Conference on Computer and Robot Vision, 2004. Proceedings..

[56]  Pamela J. Hinds,et al.  Whose job is it anyway? a study of human-robot interaction in a collaborative task , 2004 .

[57]  Sebastian Thrun,et al.  Toward a Framework for Human-Robot Interaction , 2004, Hum. Comput. Interact..

[58]  Pamela J. Hinds,et al.  Whose Job Is It Anyway? A Study of Human-Robot Interaction in a Collaborative Task , 2004, Hum. Comput. Interact..

[59]  Reinhard Moratz,et al.  Qualitative Spatial Arrangements and Natural Object Categories as a Link Between 3D-Perception and Speech , 2004, PRICAI.

[60]  Marjorie Skubic,et al.  Spatial language for human-robot dialogs , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[61]  R MurphyRobin,et al.  Moonlight in Miami , 2004 .

[62]  Kenny R. Coventry,et al.  Seeing, saying and acting: The psychological semantics of spatial prepositions , 2004 .

[63]  T. Tenbrink.  Semantics and Application of Spatial Dimensional Terms in English and German , 2005 .

[64]  Laura A. Carlson,et al.  Functional Features in Language and Space - Insights from Perception, Categorization, and Development , 2005, Functional Features in Language and Space.

[65]  Ame Rican.  I Don't Understand , 2007 .

[66]  Bernhard Nebel,et al.  Spatial Cognition IV, Reasoning, Action, Interaction , 2008 .

[67]  Reinhard Moratz,et al.  Group-based Spatial Reference in Linguistic Human-Robot Interaction , 2003 .