Computational approaches to motor learning by imitation.

Movement imitation requires a complex set of mechanisms that map an observed movement of a teacher onto one's own movement apparatus. Relevant problems include movement recognition, pose estimation, pose tracking, body correspondence, coordinate transformation from external to egocentric space, matching of observed against previously learned movement, resolution of redundant degrees-of-freedom that are unconstrained by the observation, suitable movement representations for imitation, modularization of motor control, etc. All of these topics by themselves are active research problems in computational and neurobiological sciences, such that their combination into a complete imitation system remains a daunting undertaking-indeed, one could argue that we need to understand the complete perception-action loop. As a strategy to untangle the complexity of imitation, this paper will examine imitation purely from a computational point of view, i.e. we will review statistical and mathematical approaches that have been suggested for tackling parts of the imitation problem, and discuss their merits, disadvantages and underlying principles. Given the focus on action recognition of other contributions in this special issue, this paper will primarily emphasize the motor side of imitation, assuming that a perceptual system has already identified important features of a demonstrated movement and created their corresponding spatial information. Based on the formalization of motor control in terms of control policies and their associated performance criteria, useful taxonomies of imitation learning can be generated that clarify different approaches and future research directions.

[1]  HighWire Press Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[2]  J. Piaget Play, dreams and imitation in childhood , 1951 .

[3]  M. Ciletti,et al.  The computation and theory of optimal control , 1972 .

[4]  Michael A. Arbib,et al.  Perceptual Structures and Distributed Motor Control , 1981 .

[5]  D. C. Essen,et al.  Hierarchical organization and functional streams in the visual cortex , 1983, Trends in Neurosciences.

[6]  Jean-Claude Latombe,et al.  An Approach to Automatic Robot Programming Based on Inductive Learning , 1984 .

[7]  Anthony Levas,et al.  A user-friendly high-level robot teaching system , 1984, ICRA.

[8]  Gerald DeJong,et al.  Explanation-based manipulator learning: Acquisition of planning ability through observation , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[9]  John J. Craig,et al.  Introduction to Robotics Mechanics and Control , 1986 .

[10]  Elliot Saltzman,et al.  Skilled actions: a task-dynamic approach. , 1987, Psychological review.

[11]  J. Kelso,et al.  Skilled actions: a task-dynamic approach. , 1987, Psychological review.

[12]  Alberta Maria Segre,et al.  Machine Learning of Robot Assembly Plans , 1988 .

[13]  Christopher G. Atkeson,et al.  Task-level robot learning: juggling a tennis ball more accurately , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[14]  John Baillieul,et al.  Resolution of kinematic redundancy , 1990 .

[15]  Long Ji Lin,et al.  Programming Robots Using Reinforcement Learning and Teaching , 1991, AAAI.

[16]  Claude Sammut,et al.  Learning to Fly , 1992, ML.

[17]  Katsushi Ikeuchi,et al.  Assembly task recognition with planar, curved and mechanical contacts , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[18]  Steven J. Nowlan,et al.  Mixtures of Controllers for Jump Linear and Non-Linear Plants , 1993, NIPS.

[19]  M. Tomasello,et al.  Imitative learning of actions on objects by children, chimpanzees, and enculturated chimpanzees. , 1993, Child development.

[20]  Dana H. Ballard,et al.  Recognizing teleoperated manipulations , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[21]  S. Grossberg,et al.  A Self-Organizing Neural Model of Motor Equivalent Reaching and Tool Use by a Multijoint Arm , 1993, Journal of Cognitive Neuroscience.

[22]  A. Meltzoff,et al.  Imitation, Memory, and the Representation of Persons. , 1994, Infant behavior & development.

[23]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[24]  Gillian M. Hayes,et al.  A Robot Controller Using Learning by Imitation , 1994 .

[25]  Ales Ude,et al.  Acquisition of Elementary Robot Skills from Human Demonstration , 1995 .

[26]  Yangsheng Xu,et al.  Human skill transfer: neural networks as learners and teachers , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[27]  Ashok N. Srivastava,et al.  Nonlinear gated experts for time series: discovering regimes and avoiding overfitting , 1995, Int. J. Neural Syst..

[28]  Kerstin Dautenhahn,et al.  Getting to know each other - Artificial social intelligence for autonomous robots , 1995, Robotics Auton. Syst..

[29]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[30]  Avinash C. Kak,et al.  Automatic learning of assembly tasks using a DataGlove system , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[31]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[32]  Geir Hovland,et al.  Skill acquisition from human demonstration using a hidden Markov model , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[33]  Klaus-Robert Müller,et al.  Annealed Competition of Experts for a Segmentation and Classification of Switching Dynamics , 1996, Neural Computation.

[34]  S. Schaal,et al.  A Kendama Learning Robot Based on Bi-directional Theory , 1996, Neural Networks.

[35]  Gregory Z. Grudic,et al.  Human-to-robot skill transfer using the SPORE approximation , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[36]  Günther Palm,et al.  Iterative retrieval of sparsely coded associative memory patterns , 1996, Neural Networks.

[37]  Nicholas G. Hatsopoulos,et al.  Coupling the Neural and Physical Dynamics in Rhythmic Movements , 1996, Neural Computation.

[38]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[39]  Mitsuo Kawato,et al.  Bi-directional theory approach to integration , 1996 .

[40]  F. Guenther,et al.  Neural Models for Flexible Control of Redundant Systems , 1997 .

[41]  D. Wolpert Computational approaches to motor control , 1997, Trends in Cognitive Sciences.

[42]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[43]  A. Meltzoff,et al.  Explaining Facial Imitation: A Theoretical Model. , 1997, Early development & parenting.

[44]  Stefan Schaal,et al.  Learning tasks from a single demonstration , 1997, Proceedings of International Conference on Robotics and Automation.

[45]  Mitsuo Kawato,et al.  A tennis serve and upswing learning robot based on bi-directional theory , 1998, Neural Networks.

[46]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[47]  Michael A. Arbib,et al.  Modeling parietal-premotor interactions in primate control of grasping , 1998, Neural Networks.

[48]  A. Goldman,et al.  Mirror neurons and the simulation theory of mind-reading , 1998, Trends in Cognitive Sciences.

[49]  M. Arbib,et al.  Language within our grasp , 1998, Trends in Neurosciences.

[50]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[51]  Thomas G. Dietterich Adaptive computation and machine learning , 1998 .

[52]  R. Byrne,et al.  Priming primates: Human and otherwise , 1998, Behavioral and Brain Sciences.

[53]  M. Matarić,et al.  Fixation behavior in observation and imitation of human movement. , 1998, Brain research. Cognitive brain research.

[54]  Balaraman Ravindran,et al.  Improved Switching among Temporally Abstract Actions , 1998, NIPS.

[55]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[56]  S. Schaal,et al.  Segmentation of endpoint trajectories does not imply segmented control , 1999, Experimental Brain Research.

[57]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[58]  Kerstin Dautenhahn,et al.  Of hummingbirds and helicopters: An algebraic framework for interdisciplinary studies of imitation a , 2000 .

[59]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[60]  Michael A. Arbib,et al.  Synthetic brain imaging: grasping, mirror neurons and imitation , 2000, Neural Networks.

[61]  Rieko Osu,et al.  Reciprocal excitation between biological and robotic research , 2000, SPIE Optics East.

[62]  Stefan Schaal,et al.  Learning inverse kinematics , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[63]  Aude Billard,et al.  LEARNING MOTOR SKILLS BY IMITATION: A BIOLOGICALLY INSPIRED ROBOTIC MODEL , 2001, Cybern. Syst..

[64]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[65]  Tamar Flash,et al.  Computational approaches to motor control , 2001, Current Opinion in Neurobiology.

[66]  Aude Billard,et al.  Learning human arm movements by imitation: : Evaluation of a biologically inspired connectionist architecture , 2000, Robotics Auton. Syst..

[67]  Gillian M. Hayes,et al.  Imitation as a dual-route process featuring predictive and learning components: a biologically plausible computational model , 2002 .

[68]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[69]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[70]  Chrystopher L. Nehaniv,et al.  Imitation as a Dual-Route Process Featuring Predictive and Learning Components: A Biologically Plausible Computational Model , 2002 .

[71]  Chrystopher L. Nehaniv,et al.  Imitation with ALICE: learning to imitate corresponding actions across dissimilar embodiments , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[72]  Jun Nakanishi,et al.  Learning rhythmic movements by demonstration using nonlinear oscillators , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[73]  Stefan Schaal,et al.  Computational elements of robot learning by imitation , 2002 .

[74]  Michael A. Arbib,et al.  Schema design and implementation of the grasp-related mirror neuron system , 2002, Biological Cybernetics.

[75]  K. Dautenhahn,et al.  Imitation in Animals and Artifacts , 2002 .

[76]  K. Doya,et al.  A unifying computational framework for motor control and social interaction. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[77]  R. Byrne Imitation as behaviour parsing. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[78]  Jon Rigelsford Imitation in Animals and Artifacts , 2003 .

[79]  Andrew Blake,et al.  Mathematical modelling of animate and intentional motion. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[80]  Mitsuo Kawato,et al.  A theory for cursive handwriting based on the minimization principle , 1995, Biological Cybernetics.

[81]  F. Lacquaniti,et al.  Simulation studies on the control of posture and movement in a multi-jointed limb , 2004, Biological Cybernetics.

[82]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.