A Bayesian Developmental Approach to Robotic Goal-Based Imitation Learning

A fundamental challenge in robotics today is building robots that can learn new skills by observing humans and imitating human actions. We propose a new Bayesian approach to robotic learning by imitation inspired by the developmental hypothesis that children use self-experience to bootstrap the process of intention recognition and goal-based imitation. Our approach allows an autonomous agent to: (i) learn probabilistic models of actions through self-discovery and experience, (ii) utilize these learned models for inferring the goals of human actions, and (iii) perform goal-based imitation for robotic learning and human-robot collaboration. Such an approach allows a robot to leverage its increasing repertoire of learned behaviors to interpret increasingly complex human actions and use the inferred goals for imitation, even when the robot has very different actuators from humans. We demonstrate our approach using two different scenarios: (i) a simulated robot that learns human-like gaze following behavior, and (ii) a robot that learns to imitate human actions in a tabletop organization task. In both cases, the agent learns a probabilistic model of its own actions, and uses this model for goal inference and goal-based imitation. We also show that the robotic agent can use its probabilistic model to seek human assistance when it recognizes that its inferred actions are too uncertain, risky, or impossible to perform, thereby opening the door to human-robot collaboration.

[1]  Jaime Valls Miró,et al.  Language for learning complex human-object interactions , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  A. Meltzoff,et al.  Self-experience as a mechanism for learning about others: a training study in social cognition. , 2008, Developmental psychology.

[3]  Aude Billard,et al.  Three-dimensional frames of references transformations using recurrent populations of neurons , 2005, Neurocomputing.

[4]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[5]  Rajesh P. N. Rao,et al.  A Cognitive Model of Imitative Development in Humans and Machines , 2007, Int. J. Humanoid Robotics.

[6]  Andrew N. Meltzoff,et al.  Self discovery enables robot social cognition: Are you my teacher? , 2010, Neural Networks.

[7]  A. Gopnik,et al.  Causal learning from probabilistic events in 24-month-olds: an action measure. , 2015, Developmental science.

[8]  Rajesh P. N. Rao,et al.  Imitation and Social Learning in Robots, Humans and Animals: A Bayesian model of imitation in infants and robots , 2007 .

[9]  A. Meltzoff 'Like me': a foundation for social cognition. , 2007, Developmental science.

[10]  Bernhard Schölkopf,et al.  Probabilistic movement modeling for intention inference in human–robot interaction , 2013, Int. J. Robotics Res..

[11]  Luís A. Alexandre,et al.  Artificial Neural Networks - ICANN 2007, 17th International Conference, Porto, Portugal, September 9-13, 2007, Proceedings, Part I , 2007, International Conference on Artificial Neural Networks.

[12]  Chrystopher L. Nehaniv Computation for Metaphors, Analogy, and Agents , 2000, Lecture Notes in Computer Science.

[13]  Oussama Khatib,et al.  Springer Handbook of Robotics , 2007, Springer Handbooks.

[14]  C. Breazeal,et al.  Robots that imitate humans , 2002, Trends in Cognitive Sciences.

[15]  Rajesh P. N. Rao,et al.  Imitation Learning Using Graphical Models , 2007, ECML.

[16]  Rajesh P. N. Rao,et al.  Planning and Acting in Uncertain Environments using Probabilistic Inference , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[18]  Joost N. Kok Machine Learning: ECML 2007, 18th European Conference on Machine Learning, Warsaw, Poland, September 17-21, 2007, Proceedings , 2007, ECML.

[19]  Cynthia Breazeal,et al.  Learning From and About Others: Towards Using Imitation to Bootstrap the Social Understanding of Others by Robots , 2005, Artificial Life.

[20]  C. Breazeal,et al.  Challenges in building robots that imitate people , 2002 .

[21]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[22]  Rajesh P. N. Rao,et al.  "Social" robots are psychological agents for infants: A test of gaze following , 2010, Neural Networks.

[23]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[24]  G. Mettaa,et al.  A developmental approach to visually-guided reaching in artificial systems , 1999 .

[25]  A. Meltzoff,et al.  Explaining Facial Imitation: A Theoretical Model. , 1997, Early development & parenting.

[26]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[27]  Aude Billard,et al.  A biologically inspired robotic model for learning by imitation , 2000, AGENTS '00.

[28]  K. Laland,et al.  Lessons from animal teaching. , 2008, Trends in ecology & evolution.

[29]  Sebastian Thrun,et al.  FastSLAM: a factored solution to the simultaneous localization and mapping problem , 2002, AAAI/IAAI.

[30]  Minoru Asada,et al.  Development of goal-directed gaze shift based on predictive learning , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[31]  Danica Kragic,et al.  Task modeling in imitation learning using latent variable models , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[32]  A. Cangelosi,et al.  Developmental Robotics: From Babies to Robots , 2015 .

[33]  Henk Nijmeijer,et al.  Robot Programming by Demonstration , 2010, SIMPAR.

[34]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[35]  Dieter Fox,et al.  Gambit: An autonomous chess-playing robotic system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[37]  B. Scassellati Imitation and mechanisms of joint attention: a developmental structure for building social skills on a humanoid robot , 1999 .

[38]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[39]  Rajesh P. N. Rao,et al.  Dynamic Imitation in a Humanoid Robot through Nonparametric Probabilistic Inference , 2006, Robotics: Science and Systems.

[40]  Matthew Botvinick,et al.  Goal-directed decision making in prefrontal cortex: a computational framework , 2008, NIPS.

[41]  Rajesh P. N. Rao,et al.  Learning Actions through Imitation and Exploration: Towards Humanoid Robots That Learn from Humans , 2009, Creating Brain-Like Intelligence.

[42]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[43]  P. Churchland Folk Psychology and the Explanation of Human Behavior , 1989 .

[44]  M. Matarić Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior , 1998, Trends in Cognitive Sciences.

[45]  Yukie Nagai,et al.  Staged Development of Robot Skills: Behavior Formation, Affordance Learning and Imitation with Motionese , 2015, IEEE Transactions on Autonomous Mental Development.

[46]  Rajesh P. N. Rao,et al.  Goal-Based Imitation as Probabilistic Inference over Graphical Models , 2005, NIPS.

[47]  Danica Kragic,et al.  Learning task constraints for robot grasping using graphical models , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[48]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[49]  Oliver Kroemer,et al.  Towards learning hierarchical skills for multi-phase manipulation tasks , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[50]  Il Hong Suh,et al.  A temporal Bayesian network with application to design of a proactive robotic assistant , 2012, 2012 IEEE International Conference on Robotics and Automation.

[51]  Bernhard Sendhoff,et al.  Creating Brain-Like Intelligence , 2009, Creating Brain-Like Intelligence.

[52]  Martin Davies,et al.  Mental Simulation: Evaluations and Applications - Reading in Mind and Language , 1995 .

[53]  W. Wong,et al.  On ψ-Learning , 2003 .

[54]  A. Meltzoff Understanding the Intentions of Others: Re-Enactment of Intended Acts by 18-Month-Old Children. , 1995, Developmental psychology.

[55]  Giulio Sandini,et al.  A developmental approach to visually-guided reaching in artificial systems , 1999, Neural Networks.

[56]  A. Meltzoff The 'like me' framework for recognizing and becoming an intentional agent. , 2007, Acta psychologica.

[57]  G. Csibra,et al.  Alison Gopnik Andrew N. Meltroff , 1997, Trends in Cognitive Sciences.

[58]  Terrence J Sejnowski,et al.  Foundations for a New Science of Learning , 2009, Science.

[59]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[60]  Minoru Asada,et al.  Cognitive developmental robotics as a new paradigm for the design of humanoid robots , 2001, Robotics Auton. Syst..

[61]  Aude Billard,et al.  Interferences in the Transformation of Reference Frames During a Posture Imitation Task , 2007, ICANN.