Web-Based Remote Assistance to Overcome Robot Perceptual Limitations ∗

This paper addresses the problem of overcoming visual perception limitations in service robots with remote assistance from human users. In particular, consider a scenario where a user requests the robot to perform some task that requires a perceptual ability, e.g., check if a specific mug, “my mug,” is in the lab or in an office, but the robot may not know how to recognize that object. We propose to equip the robots with the abilities to: (i) identify their own perceptual limitations, (ii) autonomously and remotely query human users for assistance, and (iii) learn new object descriptors from the interaction with humans. We successfully developed a complete initial version of our approach on our CoBot service mobile robot. The interaction with the user builds upon our previously developed semiautonomous telepresence image sharing and control. The user can now further identify the object and the robot can save the descriptor and use it in future situations. We illustrate our work with the task of learning to identify an object in the environment, and to report its presence to a user. Our ongoing work includes addressing a dynamic interaction between the robot and the remote user for visual focus of attention and different object viewing, as well as the effective storage, labeling, accessing, and sharing of multiple learned object descriptors, in particular among robots. Our goal is also to contribute the learned knowledge to crowd-robotics efforts.

[1]  Roland Siegwart,et al.  Interacting Mobile Robots on the Web , 1999 .

[2]  Francesco Mondada,et al.  KhepOnTheWeb: open access to a mobile robot on the Internet , 2000, IEEE Robotics Autom. Mag..

[3]  Erika Rogers,et al.  Cooperative Assistance for Remote Robot Supervision , 1995, Presence: Teleoperators & Virtual Environments.

[4]  Wolfram Burgard,et al.  Web interfaces for mobile robots in public places , 2000, IEEE Robotics Autom. Mag..

[5]  Ubbo Visser,et al.  RoboCup: 10 Years of Achievements and Future Challenges , 2007, AI Mag..

[6]  Manuela M. Veloso,et al.  Effective Semi-autonomous Telepresence , 2012, RoboCup.

[7]  Stephanie Rosenthal,et al.  Symbiotic-Autonomous Service Robots for User-Requested Tasks in a Multi-Floor Building , 2012, IROS 2012.

[8]  Kenneth Y. Goldberg,et al.  Desktop teleoperation via the World Wide Web , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[9]  Stephanie Rosenthal,et al.  Task Behavior and Interaction Planning for a Mobile Service Robot that Occasionally Requires Help , 2011, Automated Action Planning for Autonomous Mobile Robots.

[10]  Manuela M. Veloso,et al.  Using the Web to Interactively Learn to Find Objects , 2012, AAAI.

[11]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[12]  K. Taylor,et al.  Australia's Telerobot On The Web , 1995 .

[13]  Karen Zita Haigh,et al.  A layered architecture for office delivery robots , 1997, AGENTS '97.