Robot Programming by Demonstration

Also referred to as learning by imitation, tutelage, or apprenticeship learning, Programming by Demonstration (PbD) develops methods by which new skills can be transmitted to a robot. This book examines methods by which robots learn new skills through human guidance. Taking a practical perspective, it covers a broad range of applications, including service robots. The text addresses the challenges involved in investigating methods by which PbD is used to provide robots with a generic and adaptive model of control. Drawing on findings from robot control, human-robot interaction, applied machine learning, artificial intelligence, and developmental and cognitive psychology, the book contains a large set of didactic and illustrative examples. Practical and comprehensive machine learning source codes are available on the books companion website: http://www.programming-by-demonstration.org

[1]  J. Piaget.  Play, dreams and imitation in childhood , 1951 .

[2]  John H. R. Maunsell,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. , 1983, Journal of neurophysiology.

[3]  Jean-Claude Latombe,et al.  An Approach to Automatic Robot Programming Based on Inductive Learning , 1984 .

[4]  Anthony Levas,et al.  A user-friendly high-level robot teaching system , 1984, ICRA.

[5]  Gerald DeJong,et al.  Explanation-based manipulator learning: Acquisition of planning ability through observation , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[6]  Alberta Maria Segre,et al.  Machine Learning of Robot Assembly Plans , 1988 .

[7]  Christopher G. Atkeson,et al.  Using Local Models to Control Movement , 1989, NIPS.

[8]  Daniel Bullock,et al.  Chapter 11 Vite and Flete: Neural Modules for Trajectory Formation and Postural Control , 1989 .

[9]  Andrew W. Moore,et al.  Fast, Robust Adaptive Control by Learning only Forward Models , 1991, NIPS.

[10]  Katsushi Ikeuchi,et al.  Towards an assembly plan from observation. I. Assembly task recognition using face-contact relations (polyhedral objects) , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[11]  Ales Ude,et al.  Trajectory generation from noisy positions of object features for teaching robot paths , 1993, Robotics Auton. Syst..

[12]  Dana H. Ballard,et al.  Recognizing teleoperated manipulations , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[13]  Sheng Liu,et al.  Teaching and learning of deburring robots using neural networks , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[14]  Yangsheng Xu,et al.  Hidden Markov model approach to skill learning and its application to telerobotics , 1993, IEEE Trans. Robotics Autom..

[15]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[16]  Kerstin Dautenhahn,et al.  Getting to know each other - Artificial social intelligence for autonomous robots , 1995, Robotics Auton. Syst..

[17]  Stefan Schaal,et al.  From Isolation to Cooperation: An Alternative View of a System of Experts , 1995, NIPS.

[18]  Avinash C. Kak,et al.  Automatic learning of assembly tasks using a DataGlove system , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[19]  Katsushi Ikeuchi,et al.  A robot system that observes and replicates grasping tasks , 1995, Proceedings of IEEE International Conference on Computer Vision.

[20]  Geir Hovland,et al.  Skill acquisition from human demonstration using a hidden Markov model , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[21]  Nathan Delson,et al.  Robot programming by human demonstration: adaptation and inconsistency in constrained motion , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[22]  N. Kambhatla.  Local models and Gaussian mixture models for statistical data processing , 1996 .

[23]  Rüdiger Dillmann,et al.  Building elementary robot skills from human demonstration , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[24]  S. K. Tso,et al.  Hidden Markov model for intelligent extraction of robot trajectory command from demonstrated trajectories , 1996, Proceedings of the IEEE International Conference on Industrial Technology (ICIT'96).

[25]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[26]  Yangsheng Xu,et al.  Human action learning via hidden Markov model , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[27]  Ferdinando A. Mussa-Ivaldi,et al.  Nonlinear force fields: a distributed system of control primitives for representing and learning movements , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.

[28]  Sorin Moga,et al.  From Perception-Action Loops to Imitation Processes: A Bottom-Up Approach of Learning by Imitation , 1998, Appl. Artif. Intell..

[29]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[30]  Michael A. Arbib,et al.  Modeling parietal-premotor interactions in primate control of grasping , 1998, Neural Networks.

[31]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[32]  Aude Billard,et al.  Grounding communication in autonomous robots: An experimental study , 1998, Robotics Auton. Syst..

[33]  B. Scassellati.  Imitation and mechanisms of joint attention: a developmental structure for building social skills on a humanoid robot , 1999 .

[34]  Balaraman Ravindran,et al.  Improved Switching among Temporally Abstract Actions". In Advances in Neural Information Processing Systems , 1999 .

[35]  S. Schaal,et al.  Segmentation of endpoint trajectories does not imply segmented control , 1999, Experimental Brain Research.

[36]  J. Nadel,et al.  The evolving nature of imitation as a format for communication. , 1999 .

[37]  Aude Billard,et al.  DRAMA, a Connectionist Architecture for Control and Learning in Autonomous Robots , 1999, Adapt. Behav..

[38]  J. Mazziotta,et al.  Cortical mechanisms of human imitation. , 1999, Science.

[39]  Perry Y. Li,et al.  Passive velocity field control of mechanical manipulators , 1995, IEEE Trans. Robotics Autom..

[40]  G. Rizzolatti,et al.  Resonance behaviors and mirror neurons. , 1999, Archives italiennes de biologie.

[41]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[42]  S. Münch,et al.  Robot Programming by Demonstration (RPD) - Using Machine Learning and User Interaction Methods for the Development of Easy and Comfortable Robot Programming Systems , 2000 .

[43]  Stefan Schaal,et al.  Nonparametric Regression for Learning Nonlinear Transformations , 2000 .

[44]  W. Prinz,et al.  Compatibility between Observed and Executed Finger Movements: Comparing Symbolic, Spatial, and Imitative Cues , 2000, Brain and Cognition.

[45]  Richard A. Volz,et al.  Acquiring robust, force-based assembly skills from human demonstration , 2000, IEEE Trans. Robotics Autom..

[46]  Michael A. Arbib,et al.  Synthetic brain imaging: grasping, mirror neurons and imitation , 2000, Neural Networks.

[47]  Mohammed Yeasin,et al.  Toward automatic robot programming: learning human skill from visual data , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[48]  H. Bekkering,et al.  Imitation of gestures in children is goal-directed. , 2000, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[49]  John G. Taylor,et al.  A neural model of working memory processes in normal subjects, Parkinson's disease and schizophrenia for fMRI design and predictions , 2000, Neural Networks.

[50]  H. Kozima,et al.  A Robot that Learns to Communicate with Human Caregivers , 2001 .

[51]  Abderrahmane Kheddar,et al.  Teleoperation based on the hidden robot concept , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[52]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[53]  Tetsuo Ono,et al.  Development of an Interactive Humanoid Robot "Robovie" - An interdisciplinary approach , 2001, ISRR.

[54]  Cristina P. Santos,et al.  Control of movement time and sequential action through attractor dynamics: A simulation study demonstrating object interception and coordination , 2001 .

[55]  Aude Billard,et al.  Learning human arm movements by imitation: : Evaluation of a biologically inspired connectionist architecture , 2000, Robotics Auton. Syst..

[56]  A. Meltzoff,et al.  A PET Exploration of the Neural Mechanisms Involved in Reciprocal Imitation , 2002, NeuroImage.

[57]  Gillian M. Hayes,et al.  Imitation as a dual-route process featuring prediction and learning components: A biologically plaus , 2002 .

[58]  Yoshihiko Nakamura,et al.  Acquiring Motion Elements for Bidirectional Computation of Motion Recognition and Generation , 2002, ISER.

[59]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[60]  Aude Billard,et al.  Imitation: a Means to Enhance Learning of a Synthetic Proto-language in an Autonomous Robot , 1999 .

[61]  Katsushi Ikeuchi,et al.  Task analysis based on observing hands and objects by vision , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  Maja J. Matarić,et al.  Sensory-motor primitives as a basis for imitation: linking perception to action and biology to robotics , 2002 .

[63]  Cynthia Breazeal,et al.  Recognition of Affective Communicative Intent in Robot-Directed Speech , 2002, Auton. Robots.

[64]  Michael A. Arbib,et al.  Schema design and implementation of the grasp-related mirror neuron system , 2002, Biological Cybernetics.

[65]  Rainer Stiefelhagen,et al.  Pointing gesture recognition based on 3D-tracking of face, hands and head orientation , 2003, ICMI '03.

[66]  Yoshihiko Nakamura,et al.  Keyframe compression and decompression for time series data based on the continuous hidden Markov model , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[67]  Stefan Schaal,et al.  Reinforcement Learning for Humanoid Robotics , 2003 .

[68]  Tomomasa Sato,et al.  Robot imitation of human motion based on qualitative description from multiple measurement of human and environmental data , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[69]  Katsushi Ikeuchi,et al.  Extraction of essential interactions through multiple observations of human demonstrations , 2003, IEEE Trans. Ind. Electron..

[70]  Katsu Yamane,et al.  Dynamics Filter - concept and implementation of online motion Generator for human figures , 2000, IEEE Trans. Robotics Autom..

[71]  Monica N. Nicolescu,et al.  Natural methods for robot task learning: instructive demonstrations, generalization and practice , 2003, AAMAS '03.

[72]  S. Bocionek,et al.  Robot programming by Demonstration (RPD): Supporting the induction by human interaction , 1996, Machine Learning.

[73]  Gordon Cheng,et al.  Learning tasks from observation and practice , 2004, Robotics Auton. Syst..

[74]  Jianwei Zhang,et al.  Self-valuing learning and generalization with application in visually guided grasping of complex objects , 2004, Robotics Auton. Syst..

[75]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[76]  Rüdiger Dillmann,et al.  Teaching and learning of robot tasks via observation of human performance , 2004, Robotics Auton. Syst..

[77]  Aaron Hertzmann,et al.  Style-based inverse kinematics , 2004, SIGGRAPH 2004.

[78]  Masayuki Inaba,et al.  Development of an imitation behavior in humanoid Kenta with reinforcement learning algorithm based on the attention during imitation , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[79]  Minoru Asada,et al.  Periodic nonlinear principal component neural networks for humanoid motion segmentation, generalization, and generation , 2004, ICPR 2004.

[80]  Stefano Caselli,et al.  Leveraging on a virtual environment for robot programming by demonstration , 2004, Robotics Auton. Syst..

[81]  Yasuo Kuniyoshi,et al.  Embodied basis of invariant features in execution and perception of whole-body dynamic actions - knacks and focuses of Roll-and-Rise motion , 2004, Robotics Auton. Syst..

[82]  Jun Tani,et al.  Joint attention between a humanoid robot and users in imitation game , 2004 .

[83]  Rajesh P. N. Rao,et al.  Robotic imitation from human motion capture using Gaussian processes , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[84]  Peter Ford Dominey,et al.  Robot command, interrogation and teaching via social interaction , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[85]  Cynthia Breazeal,et al.  Learning From and About Others: Towards Using Imitation to Bootstrap the Social Understanding of Others by Robots , 2005, Artificial Life.

[86]  Rüdiger Dillmann,et al.  Learning sequential constraints of tasks from user demonstrations , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[87]  Stefano Caselli,et al.  Trajectory clustering and stochastic approximation for robot programming by demonstration , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[88]  Danica Kragic,et al.  Grasp Recognition for Programming by Demonstration , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[89]  Masayuki Inaba,et al.  Intent imitation using wearable motion capturing system with on-line teaching of task attention , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[90]  Rüdiger Dillmann,et al.  Towards Cognitive Robots: Building Hierarchical Task Representations of Manipulations from Human Demonstration , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[91]  Frédéric Kaplan,et al.  Learning to Interpret Pointing Gestures: Experiments with Four-Legged Autonomous Robots , 2005, Biomimetic Neural Learning for Intelligent Robots.

[92]  Stefan Schaal,et al.  Incremental Online Learning in High Dimensions , 2005, Neural Computation.

[93]  Monica N. Nicolescu,et al.  Task Learning through Imitation and Human-robot Interaction , 2005 .

[94]  Aude Billard,et al.  Recognition and reproduction of gestures using a probabilistic framework combining PCA, ICA and HMM , 2005, ICML.

[95]  Aude Billard,et al.  Goal-Directed Imitation in a Humanoid Robot , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[96]  Aude Billard,et al.  Dynamic updating of distributed neural representations using forward models , 2006, Biological Cybernetics.

[97]  Danica Kragic,et al.  Learning Task Models from Multiple Human Demonstrations , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[98]  Aude Billard,et al.  Learning Dynamical System Modulation for Constrained Reaching Tasks , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[99]  Ignazio Infantino,et al.  A cognitive framework for imitation learning , 2006, Robotics Auton. Syst..

[100]  Estela Bicho,et al.  Goal-directed imitation for robots: A bio-inspired approach to action understanding and skill learning , 2006, Robotics Auton. Syst..

[101]  Andrea Lockerd Thomaz,et al.  Using perspective taking to learn from ambiguous demonstrations , 2006, Robotics Auton. Syst..

[102]  Aude Billard,et al.  Biologically Inspired Multimodal Integration: Interferences in a Human-Robot Interaction Game , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[103]  Aude Billard,et al.  Teaching a Humanoid Robot to Recognize and Reproduce Social Cues , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[104]  Stefano Caselli,et al.  Robust trajectory learning and approximation for robot programming by demonstration , 2006, Robotics Auton. Syst..

[105]  Michael A. Arbib,et al.  Mirror neurons and imitation: A computationally guided review , 2006, Neural Networks.

[106]  Gordon Cheng,et al.  Dexterous Skills Transfer by Extending Human Body Schema to a Robotic Hand , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[107]  Matthew W. Hoffman,et al.  A probabilistic model of gaze imitation and shared attention , 2006, Neural Networks.

[108]  Yoon-Kwon Hwang,et al.  Self-learning control of cooperative motion for a humanoid robot , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[109]  Chrystopher L. Nehaniv,et al.  Teaching robots by moulding behavior and scaffolding the environment , 2006, HRI '06.

[110]  Jun Tani,et al.  Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model , 2006, Neural Networks.

[111]  Masaki Ogino,et al.  Interaction rule learning with a human partner based on an imitation faculty with a simple visuo-motor mapping , 2006, Robotics Auton. Syst..

[112]  Aude Billard,et al.  Discriminative and adaptive imitation in uni-manual and bi-manual tasks , 2006, Robotics Auton. Syst..

[113]  Yoshihiko Nakamura,et al.  Stochastic Model of Imitating a New Observed Motion Based on the Acquired Motion Primitives , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[114]  Tony Belpaeme,et al.  A computational model of intention reading in imitation , 2006, Robotics Auton. Syst..

[115]  Tamim Asfour,et al.  Imitation Learning of Dual-Arm Manipulation Tasks in Humanoid Robots , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[116]  Masayuki Inaba,et al.  Situation Recognition and Behavior Induction based on Geometric Symbol Representation of Multimodal Sensorimotor Patterns , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[117]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[118]  Raymond H. Cuijpers,et al.  Goals and means in action observation: A computational approach , 2006, Neural Networks.

[119]  Aude Billard,et al.  Parallel and distributed neural models of the ideomotor principle: An investigation of imitative cortical pathways , 2006, Neural Networks.

[120]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[121]  Aude Billard,et al.  Reinforcement learning for imitating constrained reaching movements , 2007, Adv. Robotics.

[122]  Aude Billard,et al.  What is the Teacher"s Role in Robot Programming by Demonstration? - Toward Benchmarks for Improved Learning , 2007 .

[123]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[124]  Aude Billard,et al.  Learning of Gestures by Imitation in a Humanoid Robot , 2007 .

[125]  Rüdiger Dillmann,et al.  Incremental Learning of Tasks From User Demonstrations, Past Experiences, and Vocal Comments , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[126]  Chrystopher L. Nehaniv,et al.  Correspondence Mapping Induced State and Action Metrics for Robotic Imitation , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[127]  Yoshihiko Nakamura,et al.  Mimesis Scheme using a Monocular Vision System on a Humanoid Robot , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.