Learning Action Primitives

The use of action primitives plays an important role in modeling actions. Action primitives are motivated not only by neurobiological findings, they also allow an efficient and effective action modeling from an information-theoretic viewpoint. Different approaches for modeling action primitives have been proposed. This chapter overviews the recent approaches for learning and modeling action primitives for human and robot action and describes common approaches such as stochastic methods and dynamical systems approaches. Active research questions in the field are introduced, including temporal segmentation, dimensionality reduction, and the integration of action primitives into complex behaviors.

[1]  Manuel Lopes,et al.  Learning Object Affordances: From Sensory--Motor Coordination to Imitation , 2008, IEEE Transactions on Robotics.

[2]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  A. Wohlschlager,et al.  Action generation and action perception in imitation : An instance of the ideomotor principle , 2003 .

[4]  Katsushi Ikeuchi,et al.  Toward automatic robot instruction from perception-recognizing a grasp from observation , 1993, IEEE Trans. Robotics Autom..

[5]  Danica Kragic,et al.  Online task recognition and real-time adaptive assistance for computer-aided machine control , 2006, IEEE Transactions on Robotics.

[6]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[7]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[8]  R. Byrne,et al.  Priming primates: Human and otherwise , 1998, Behavioral and Brain Sciences.

[9]  Danica Kragic,et al.  Learning Actions from Observations , 2010, IEEE Robotics & Automation Magazine.

[10]  Katsushi Ikeuchi,et al.  Towards an assembly plan from observation. I. Assembly task recognition using face-contact relations (polyhedral objects) , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[11]  J Rosen,et al.  Hidden Markov models of minimally invasive surgery. , 2000, Studies in health technology and informatics.

[12]  Avinash C. Kak,et al.  Automatic learning of assembly tasks using a DataGlove system , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[13]  Dana Kulic,et al.  Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains , 2008, Int. J. Robotics Res..

[14]  Hiroshi Kimura,et al.  Adaptive Motion of Animals and Machines , 2005 .

[15]  Darwin G. Caldwell,et al.  Learning and Reproduction of Gestures by Imitation , 2010, IEEE Robotics & Automation Magazine.

[16]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[17]  Yoji Yamada,et al.  An adaptive visual attentive tracker for human communicational behaviors using HMM-based TD learning with new State distinction capability , 2005, IEEE Transactions on Robotics.

[18]  Aude Billard,et al.  Using reinforcement learning to adapt an imitation task , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  J. Sinapov,et al.  Detecting the functional similarities between tools using a hierarchical representation of outcomes , 2008, 2008 7th IEEE International Conference on Development and Learning.

[20]  E Bizzi,et al.  Motor learning through the combination of primitives. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[21]  Li Wang,et al.  Human Action Segmentation and Recognition Using Discriminative Semi-Markov Models , 2011, International Journal of Computer Vision.

[22]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[23]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[24]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[25]  Wolfram Burgard,et al.  Learning Motion Patterns of People for Compliant Robot Motion , 2005, Int. J. Robotics Res..

[26]  Naoto Iwahashi,et al.  Motion recognition and generation by combining reference-point-dependent probabilistic models , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Stefan Schaal,et al.  Computational approaches to motor learning by imitation. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[28]  Aude Billard,et al.  Handbook of Robotics Chapter 59 : Robot Programming by Demonstration , 2007 .

[29]  Daniel Grest,et al.  Using Hidden Markov Models for Recognizing Action Primitives in Complex Actions , 2007, SCIA.

[30]  Wei Liang,et al.  Discriminative human action recognition in the learned hierarchical manifold space , 2010, Image Vis. Comput..

[31]  A. Meltzoff 1 Imitation and Other Minds: The "Like Me" Hypothesis , 2005 .

[32]  Trevor F. Cox,et al.  Metric multidimensional scaling , 2000 .

[33]  Stefan Schaal,et al.  Applying the Episodic Natural Actor-Critic Architecture to Motor Primitive Learning , 2007, ESANN.

[34]  M. Matarić,et al.  Evaluation Metrics and Results of Human Arm Movement Imitation , 2000 .

[35]  Katsu Yamane,et al.  Primitive communication based on motion recognition and generation with hierarchical mimesis model , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[36]  Stefan Schaal,et al.  Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning , 2002, Applied Intelligence.

[37]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  Maya Cakmak,et al.  From primitive behaviors to goal-directed behavior using affordances , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Gordon Cheng,et al.  Synthesizing goal-directed actions from a library of example movements , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[40]  U. Goswami Blackwell handbook of childhood cognitive development , 2002 .

[41]  S. Schaal Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[42]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[43]  Stefan Schaal,et al.  Reinforcement Learning for Operational Space Control , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[44]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[45]  José Santos-Victor,et al.  Visual learning by imitation with motor representations , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[46]  Ales Ude,et al.  Programming full-body movements for humanoid robots by observation , 2004, Robotics Auton. Syst..

[47]  Aude Billard,et al.  Learning of Gestures by Imitation in a Humanoid Robot , 2007 .

[48]  Maja J. Matarić,et al.  Behavior-Based Segmentation of Demonstrated Task , 2006 .

[49]  Shigeki Sugano,et al.  Open-End Human Robot Interaction from the Dynamical Systems Perspective: Mutual Adaptation and Incremental Learning , 2004, International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems.

[50]  Rüdiger Dillmann,et al.  Incremental Learning of Tasks From User Demonstrations, Past Experiences, and Vocal Comments , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[51]  G. Metta Development in artificial systems , 2001 .

[52]  Sethu Vijayakumar,et al.  Latent spaces for dynamic movement primitives , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[53]  Rüdiger Dillmann,et al.  Learning Robot Behaviour and Skills Based on Human Demonstration and Advice: The Machine Learning Paradigm , 2000 .

[54]  Maya Cakmak,et al.  To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control , 2007, Adapt. Behav..

[55]  Aude Billard,et al.  Active Teaching in Robot Programming by Demonstration , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[56]  Danica Kragic,et al.  Visual object-action recognition: Inferring object affordances from human demonstration , 2011, Comput. Vis. Image Underst..

[57]  Stefan Schaal,et al.  Incremental Online Learning in High Dimensions , 2005, Neural Computation.

[58]  Yiannis Demiris,et al.  Distributed, predictive perception of actions: a biologically inspired robotics architecture for imitation and learning , 2003, Connect. Sci..

[59]  Steven Lemm,et al.  A Dynamic HMM for On-line Segmentation of Sequential Data , 2001, NIPS.

[60]  C. Breazeal,et al.  Robots that imitate humans , 2002, Trends in Cognitive Sciences.

[61]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[62]  A. Meltzoff Imitation as a mechanism of social cognition: Origins of empathy, theory of mind, and the representation of action. , 2007 .

[63]  Dana Kulic,et al.  Comparative study of representations for segmentation of whole body human motion data , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[64]  C. Heyes Causes and consequences of imitation , 2001, Trends in Cognitive Sciences.

[65]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[66]  Rüdiger Dillmann,et al.  Feature Set Selection and Optimal Classifier for Human Activity Recognition , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[67]  Maja J. Mataric,et al.  Performance-Derived Behavior Vocabularies: Data-Driven Acquisition of Skills from Motion , 2004, Int. J. Humanoid Robotics.

[68]  Pradeep K. Khosla,et al.  Predictive Robot Programming: Theoretical and Experimental Analysis , 2004, Int. J. Robotics Res..

[69]  Katsushi Ikeuchi,et al.  Toward automatic robot instruction from perception-temporal segmentation of tasks from human hand motion , 1993, IEEE Trans. Robotics Autom..

[70]  Christiaan J. J. Paredis,et al.  Interactive Multimodal Robot Programming , 2005, Int. J. Robotics Res..

[71]  C. Heyes,et al.  What Is the Significance of Imitation in Animals , 2000 .

[72]  Katsushi Ikeuchi,et al.  A sensor fusion approach for recognizing continuous human grasping sequences using hidden Markov models , 2005, IEEE Transactions on Robotics.

[73]  Y. Nakamura,et al.  Unsupervised probabilistic segmentation of motion data for mimesis modeling , 2005, ICAR '05. Proceedings., 12th International Conference on Advanced Robotics, 2005..

[74]  Rodney A. Brooks,et al.  Humanoid robots , 2002, CACM.

[75]  Cynthia Breazeal,et al.  Improvements on action parsing and action interpolation for learning through demonstration , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[76]  Christiaan J. J. Paredis,et al.  Interactive multi-modal robot programming , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[77]  S Hurley,et al.  Perspectives on Imitation , 2004 .

[78]  Yoshihiko Nakamura,et al.  Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[79]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[80]  R. Shaw,et al.  Perceiving, Acting and Knowing : Toward an Ecological Psychology , 1978 .

[81]  Giulio Sandini,et al.  Learning about objects through action - initial steps towards artificial cognition , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[82]  Stefano Caselli,et al.  Leveraging on a virtual environment for robot programming by demonstration , 2004, Robotics Auton. Syst..

[83]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[84]  Aude Billard,et al.  Discriminative and adaptive imitation in uni-manual and bi-manual tasks , 2006, Robotics Auton. Syst..

[85]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..

[86]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[87]  Gregory D. Hager,et al.  Human-Machine Collaborative Systems for Microsurgical Applications , 2005, Int. J. Robotics Res..

[88]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[89]  Rüdiger Dillmann,et al.  Teaching and learning of robot tasks via observation of human performance , 2004, Robotics Auton. Syst..

[90]  Aude Billard,et al.  Learning Non-linear Multivariate Dynamics of Motion in Robotic Manipulators , 2011, Int. J. Robotics Res..

[91]  Danica Kragic,et al.  Grasp Recognition for Programming by Demonstration , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[92]  Jun Tani,et al.  On-line Imitative Interaction with a Humanoid Robot Using a Dynamic Neural Network Model of a Mirror System , 2004, Adapt. Behav..

[93]  Danica Kragic,et al.  Interactive grasp learning based on human demonstration , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[94]  M. Matarić,et al.  Behavior-Based Segmentation of Demonstrated Tasks , 2006 .

[95]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[96]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[97]  Maja J. Mataric,et al.  A spatio-temporal extension to Isomap nonlinear dimension reduction , 2004, ICML.

[98]  J. J. Gibson The theory of affordances , 1977 .

[99]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[100]  Betty J. Mohler,et al.  Learning perceptual coupling for motor primitives , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[101]  Kilian Q. Weinberger,et al.  Learning a kernel matrix for nonlinear dimensionality reduction , 2004, ICML.

[102]  Suguru Arimoto,et al.  Learning control for robot tasks under geometric endpoint constraints , 1995, IEEE Trans. Robotics Autom..

[103]  Gregory D. Hager,et al.  Motion generation of robotic surgical tasks: Learning from expert demonstrations , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[104]  Naoto Iwahashi,et al.  Learning object-manipulation verbs for human-robot communication , 2007, WMISI '07.

[105]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[106]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[107]  Hideki Kozima,et al.  Emergence of imitation mediated by objects , 2002 .

[108]  Yangsheng Xu,et al.  Human action learning via hidden Markov model , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[109]  Maja J. Mataric,et al.  Automated Derivation of Primitives for Movement Classification , 2000, Auton. Robots.

[110]  Danica Kragic,et al.  Learning task constraints for robot grasping using graphical models , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[111]  Jivko Sinapov,et al.  Toward interactive learning of object categories by a robot: A case study with container and non-container objects , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[112]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[113]  Dana Kulic,et al.  Incremental on-line hierarchical clustering of whole body motion patterns , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[114]  Dana Kulic,et al.  Online Segmentation and Clustering From Continuous Observation of Whole Body Motions , 2009, IEEE Transactions on Robotics.