Combining world and interaction models for human-robot collaborations

As robotic technologies mature, we can imagine an increasing number of applications in which robots could soon prove to be useful in unstructured human environments. Many of those applications require a natural interface between the robot and untrained human users or are possible only in a human-robot collaborative scenario. In this paper, we study an example of such scenario in which a visually impaired person and a robotic “guide” collaborate in an unfamiliar environment. We then analyze how the scenario can be realized through language- and gesture-based human-robot interaction, combined with semantic spatial understanding and reasoning, and propose an integration of semantic world model with language and gesture models for several collaboration modes. We believe that this way practical robotic applications can be achieved in human environments with the use of currently available technology.

[1]  Patric Jensfelt,et al.  Active Visual Object Search in Unknown Environments Using Uncertain Semantics , 2013, IEEE Transactions on Robotics.

[2]  Luke S. Zettlemoyer,et al.  Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions , 2013, TACL.

[3]  Andrea Lockerd Thomaz,et al.  Effects of nonverbal communication on efficiency and robustness in human-robot teamwork , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Andrzej Pronobis,et al.  Semantic Mapping with Mobile Robots , 2011 .

[5]  Barbara Caputo,et al.  COLD: The CoSy Localization Database , 2009, Int. J. Robotics Res..

[6]  Wolfram Burgard,et al.  Conceptual spatial representations for indoor mobile robots , 2008, Robotics Auton. Syst..

[7]  Dieter Fox,et al.  Hierarchical Matching Pursuit for Image Classification: Architecture and Fast Algorithms , 2011, NIPS.

[8]  Marc Hanheide,et al.  Exploiting Probabilistic Knowledge under Uncertain Sensing for Efficient Robot Behaviour , 2011, IJCAI.

[9]  Roland Siegwart,et al.  Bayesian space conceptualization and place classification for semantic maps in mobile robotics , 2008, Robotics Auton. Syst..

[10]  Henrik I. Christensen,et al.  Bringing Together Human and Robotic Environment Representations - A Pilot Study , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Ben J. A. Kröse,et al.  BIRON, where are you? Enabling a robot to learn new places in a real home environment by integrating spoken dialog and visual localization , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Luke S. Zettlemoyer,et al.  Learning to Parse Natural Language Commands to a Robot Control System , 2012, ISER.

[13]  E. Guizzo,et al.  The rise of the robot worker , 2012, IEEE Spectrum.

[14]  Mark Steedman,et al.  Lexical Generalization in CCG Grammar Induction for Semantic Parsing , 2011, EMNLP.

[15]  Mark Steedman,et al.  Inducing Probabilistic CCG Grammars from Logical Form with Higher-Order Unification , 2010, EMNLP.

[16]  Luke S. Zettlemoyer,et al.  A Joint Model of Language and Perception for Grounded Attribute Learning , 2012, ICML.

[17]  Mark Steedman,et al.  The syntactic process , 2004, Language, speech, and communication.

[18]  Dieter Fox,et al.  RGB-D Object Recognition: Features, Algorithms, and a Large Scale Benchmark , 2013, Consumer Depth Cameras for Computer Vision.

[19]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[20]  Nick Hawes,et al.  Engineering intelligent information-processing systems with CAST , 2010, Adv. Eng. Informatics.

[21]  Vladimir A. Kulyukin,et al.  On natural language dialogue with assistive robots , 2006, HRI '06.

[22]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision , 2013, Advances in Computer Vision and Pattern Recognition.

[23]  Stephanie Rosenthal,et al.  CoBots: Collaborative robots servicing multi-floor buildings , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  S. Lauritzen,et al.  Chain graph models and their causal interpretations , 2002 .

[25]  Moritz Tenorth,et al.  KNOWROB-MAP - knowledge-linked semantic object maps , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[26]  Moritz Tenorth,et al.  A unified representation for reasoning about robot actions, processes, and their effects on objects , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Patric Jensfelt,et al.  Large-scale semantic mapping and reasoning with heterogeneous modalities , 2012, 2012 IEEE International Conference on Robotics and Automation.

[28]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[29]  Cipriano Galindo,et al.  Multi-hierarchical semantic maps for mobile robotics , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Patric Jensfelt,et al.  Representing Spatial Knowledge in Mobile Cognitive Systems , 2010 .

[31]  James J. Little,et al.  Curious George: An attentive semantic robot , 2008, Robotics Auton. Syst..

[32]  Matthias Scheutz,et al.  Tell me when and why to do it! Run-time planner model updates via natural language instruction , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[33]  Alessio Malizia,et al.  The artificiality of natural user interfaces , 2012, CACM.

[34]  Matthew R. Walter,et al.  Approaching the Symbol Grounding Problem with Probabilistic Graphical Models , 2011, AI Mag..