Modeling dynamic spatial relations with global properties for natural language-based human-robot interaction

We present a methodology for the representation of dynamic spatial relations (DSRs) with global properties as part of an approach for enabling robots to follow natural language commands from non-expert users, with particular focus on the development of spatial language primitives. Our approach to modeling DSRs is based on related research in the fields of linguistics, cognitive science, and neuroscience, and contributes novel extensions to the semantic field model of spatial prepositions. We describe novel representations of the DSRs for “to”, “through”, and “around”, discuss their applicability in path classification scenarios, and provide implementation details of path generation routines instantiating these DSRs for use in robot task planning. The paper concludes with an evaluation of our robot architecture implemented on a simulated mobile robot in a 2D home environment.

[1]  Manuela M. Veloso,et al.  Using dialog and human observations to dictate tasks to a learning robot assistant , 2008, Intell. Serv. Robotics.

[2]  Laura A. Carlson,et al.  Formulating Spatial Descriptions across Various Dialogue Contexts , 2009, Spatial Language and Dialogue.

[3]  M. Matarić,et al.  Task Learning Through Imitation and Human-Robot Interaction , 2004 .

[4]  Ioannis Iossifidis,et al.  Natural human-robot interaction through spatial language: A Dynamic Neural Field approach , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[5]  John D. Kelleher,et al.  Towards a Cognitive System that Can Recognize Spatial Regions Based on Context , 2012, AAAI.

[6]  Matthias Scheutz,et al.  Learning actions from human-robot dialogues , 2011, 2011 RO-MAN.

[7]  Hadas Kress-Gazit,et al.  Translating Structured English to Robot Controllers , 2008, Adv. Robotics.

[8]  A. Wierzbicka,et al.  Semantics and cognition. , 2006, Wiley interdisciplinary reviews. Cognitive science.

[9]  Maya Cakmak,et al.  Towards grounding concepts for transfer in goal learning from demonstration , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[10]  Monica N. Nicolescu,et al.  Task Learning through Imitation and Human-robot Interaction , 2005 .

[11]  Deb K. Roy,et al.  Learning visually grounded words and syntax for a scene description task , 2002, Comput. Speech Lang..

[12]  Marjorie Skubic,et al.  Spatial language for human-robot dialogs , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  J O'Keefe,et al.  Vector grammar, places, and the functional role of the spatial prepositions in English , 2001 .

[14]  Stefanie Tellex,et al.  Toward understanding natural language directions , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[15]  Joelle Pineau,et al.  Development and Validation of a Robust Speech Interface for Improved Human-Robot Interaction , 2009, Int. J. Soc. Robotics.

[16]  Matthew R. Walter,et al.  Approaching the Symbol Grounding Problem with Probabilistic Graphical Models , 2011, AI Mag..

[17]  Luke S. Zettlemoyer,et al.  Learning to Parse Natural Language Commands to a Robot Control System , 2012, ISER.

[18]  Jürgen Bohnemeyer,et al.  The unique vector constraint: The impact of direction changes on the linguistic segmentation of motion events. , 2003 .

[19]  Maja J. Mataric,et al.  Using Spatial Language to Guide and Instruct Robots in Household Environments , 2012, AAAI Fall Symposium: Robots Learning Interactively from Human Teachers.

[20]  B. Landau,et al.  “What” and “where” in spatial language and spatial cognition , 1993 .

[21]  Maja J. Mataric,et al.  Using semantic fields to model dynamic spatial relations in a robot architecture for natural language instruction of service robots , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  John E. Laird,et al.  Learning Grounded Language through Situated Interactive Instruction , 2012, AAAI Fall Symposium: Robots Learning Interactively from Human Teachers.

[23]  Leonard Talmy,et al.  The fundamental system of spatial schemas in language , 2005 .

[24]  Paul R. Cohen,et al.  A framework for recognizing and executing verb phrases , 2011 .

[25]  Rüdiger Dillmann,et al.  Incremental Learning of Tasks From User Demonstrations, Past Experiences, and Vocal Comments , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  John D. Kelleher,et al.  Applying Computational Models of Spatial Prepositions to Visually Situated Dialog , 2009, CL.

[27]  Beate Hamp,et al.  The Fundamental System of Spatial Schemas in Language , 2022 .

[28]  T. Belpraeme,et al.  Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions , 2006 .