A Tangible Interface for Transferring Skills

Our research focuses on exploring new modalities to make robots acquire skills in a fast and user-friendly manner. In this work we present a novel active interface with perception and projection capabilities for simplifying the skill transfer process. The interface allows humans and robots to interact with each other in the same environment, with respect to visual feedback. During the learning process, the real workspace is used as a tangible interface for helping the user to better understand what the robot has learned up to then, to display information about the task or to get feedback and guidance. Thus, the user is able to incrementally visualize and assess the learner’s state and, at the same time, focus on the skill transfer without disrupting the continuity of the teaching interaction. We also propose a proof-of-concept, as a core element of the architecture, based on an experimental setting where a pico-projector and an rgb-depth sensor are mounted onto the end-effector of a 7-DOF robotic arm.

[1]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[2]  Ramesh Raskar,et al.  Modern approaches to augmented reality , 2005, SIGGRAPH Courses.

[3]  Darwin G. Caldwell,et al.  Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Pattie Maes,et al.  LuminAR: portable robotic augmented reality interface design and prototype , 2010, UIST '10.

[5]  Abderrahmane Kheddar,et al.  Teleoperation based on the hidden robot concept , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[6]  Ramesh Raskar,et al.  Modern approaches to augmented reality , 2005, SIGGRAPH Courses.

[7]  Sriram Subramanian,et al.  Talking about tactile experiences , 2013, CHI.

[8]  B. Rogoff,et al.  Everyday Cognition: Development in Social Context , 1999 .

[9]  Aude Billard,et al.  What is the Teacher"s Role in Robot Programming by Demonstration? - Toward Benchmarks for Improved Learning , 2007 .

[10]  Johannes Schöning,et al.  Map torchlight: a mobile augmented reality camera projector unit , 2009, CHI Extended Abstracts.

[11]  Carme Torras,et al.  3D modelling of leaves from color and ToF data for robotized plant measuring , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Scott R. Klemmer,et al.  How bodies matter: five themes for interaction design , 2006, DIS '06.

[13]  Tovi Grossman,et al.  PenLight: combining a mobile projector and a digital pen for dynamic visual overlay , 2009, CHI.

[14]  Andrea Lockerd Thomaz,et al.  Teachable robots: Understanding human teaching behavior to build more effective robot learners , 2008, Artif. Intell..

[15]  Guido Bugmann,et al.  Mobile robot programming using natural language , 2002, Robotics Auton. Syst..

[16]  Manuela M. Veloso,et al.  Confidence-Based Multi-Robot Learning from Demonstration , 2010, Int. J. Soc. Robotics.

[17]  Manfred Tscheligi,et al.  Proceedings of the international conference on Advances in computer entertainment technology , 2007, ACE 2008.

[18]  Michel F. Sanner,et al.  Tangible augmented interfaces for structural molecular biology , 2005, IEEE Computer Graphics and Applications.

[19]  P. Greenfield.  A theory of the teacher in the learning activities of everyday life. , 1984 .

[20]  Masanori Sugimoto,et al.  VisiCon: a robot control interface for visualizing manipulation using a handheld projector , 2007, ACE '07.

[21]  Chris Harrison,et al.  OmniTouch: wearable multitouch interaction everywhere , 2011, UIST.

[22]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[23]  Pradeep K. Khosla,et al.  A Multi-Agent System for Programming Robotic Agents by Human Demonstration , 1998 .

[24]  S. Bocionek,et al.  Robot programming by Demonstration (RPD): Supporting the induction by human interaction , 1996, Machine Learning.

[25]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Stefano Caselli,et al.  Grasp recognition in virtual reality for robot pregrasp planning by demonstration , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[27]  Katsushi Ikeuchi,et al.  Extraction of essential interactions through multiple observations of human demonstrations , 2003, IEEE Trans. Ind. Electron..

[28]  Nathan Delson,et al.  Robot programming by human demonstration: the use of human variation in identifying obstacle free trajectories , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[29]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[30]  Jun Tani,et al.  Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model , 2006, Neural Networks.

[31]  Alberta Maria Segre,et al.  Machine Learning of Robot Assembly Plans , 1988 .

[32]  Danica Kragic,et al.  Learning Task Models from Multiple Human Demonstrations , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[33]  Paul Dourish,et al.  Where the action is , 2001 .

[34]  D. Kendall.  A Survey of the Statistical Theory of Shape , 1989 .

[35]  J. Y. S. Luh,et al.  Resolved-acceleration control of mechanical manipulators , 1980 .

[36]  Gerard Jounghyun Kim,et al.  Robots with projectors: An alternative to anthropomorphic HRI , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[37]  Monica N. Nicolescu,et al.  Natural methods for robot task learning: instructive demonstrations, generalization and practice , 2003, AAMAS '03.

[38]  Seiji Yamada,et al.  Extending Commands Embedded in Actions for Human-Robot Cooperative Tasks , 2010, Int. J. Soc. Robotics.

[39]  Christoph Walter,et al.  Towards safe physical human-robot collaboration: A projection-based safety system , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.