Cognition Reversed : Robot Learning from Demonstration

The work presented in this thesis investigates techniques for learning from demonstration (LFD). LFD is a well established approach to robot learning, where a teacher demonstrates a behavior to a r ...

[1]  J. Konorski.  Integrative activity of the brain , 1967 .

[2]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[3]  Christian Balkenius,et al.  Cognitive modeling with context sensitive reinforcement learning , 2004 .

[4]  V. Braitenberg.  Vehicles, Experiments in Synthetic Psychology , 1984 .

[5]  G. Rizzolatti,et al.  Motor facilitation during action observation: a magnetic stimulation study. , 1995, Journal of neurophysiology.

[6]  S. C. Johnson.  Hierarchical clustering schemes , 1967, Psychometrika.

[7]  Aude Billard,et al.  Learning human arm movements by imitation: : Evaluation of a biologically inspired connectionist architecture , 2000, Robotics Auton. Syst..

[8]  K. Dautenhahn,et al.  The correspondence problem , 2002 .

[9]  A. Georgopoulos,et al.  Modular organization of directionally tuned cells in the motor cortex: Is there a short-range order? , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[10]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[11]  Maja J. Matarić,et al.  A framework for learning from demonstration, generalization and practice in human-robot domains , 2003 .

[12]  B. Rohrer,et al.  A learning and control approach based on the human neuromotor system , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[13]  Henry Lieberman,et al.  Watch what I do: programming by demonstration , 1993 .

[14]  Maja J. Matarić,et al.  Behavior-Based Segmentation of Demonstrated Task , 2006 .

[15]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[16]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[17]  C. Gross.  Genealogy of the “Grandmother Cell” , 2002, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[18]  Karl J. Friston,et al.  Free-energy and the brain , 2007, Synthese.

[19]  John Demiris,et al.  Movement imitation mechanisms in robots and humans , 1999 .

[20]  Thomas Hellström,et al.  Autonomous Forest Machines: Techniques and Algorithms for Unmanned Vehicles , 2008 .

[21]  Leslie G. Ungerleider,et al.  Neural correlates of category-specific knowledge , 1996, Nature.

[22]  Karl J. Friston,et al.  Predictive coding: an account of the mirror neuron system , 2007, Cognitive Processing.

[23]  Yasuo Kuniyoshi,et al.  Deferred imitation of human head movements by an active stereo vision head , 1997, Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN'97 SENDAI.

[24]  J. Hawkins,et al.  On Intelligence , 2004 .

[25]  Y. Demiris,et al.  From motor babbling to hierarchical learning by imitation: a robot developmental pathway , 2005 .

[26]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[27]  Charles E. Connor.  Neuroscience: Friends and grandmothers , 2005, Nature.

[28]  L. S. Vygotskiĭ,et al.  Mind in society : the development of higher psychological processes , 1978 .

[29]  S. Wise,et al.  Changes in motor cortical activity during visuomotor adaptation , 1998, Experimental Brain Research.

[30]  R A Brooks,et al.  New Approaches to Robotics , 1991, Science.

[31]  Nils J. Nilsson,et al.  APPLICATION OF INTELLIGENT AUTOMATA TO RECONNAISSANCE. , 1967 .

[32]  Fabien Lagriffoul,et al.  Activity Recognition Based on Intra and Extra Manipulation of Everyday Objects , 2007, UCS.

[33]  E. Rolls,et al.  View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex. , 1998, Cerebral cortex.

[34]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[36]  Rainer Bischoff,et al.  Robotic Visions to 2020 and beyond -- The strategic research agenda for robotics in Europe , 2009 .

[37]  Kerstin Dautenhahn,et al.  Of hummingbirds and helicopters: An algebraic framework for interdisciplinary studies of imitation a , 2000 .

[38]  Karl J. Friston.  Functional integration and inference in the brain , 2002, Progress in Neurobiology.

[39]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[40]  B. Rohrer,et al.  S-Learning: A Biomimetic Algorithm for Learning, Memory, and Control in Robots , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[41]  S. Münch,et al.  Robot Programming by Demonstration (RPD) - Using Machine Learning and User Interaction Methods for the Development of Easy and Comfortable Robot Programming Systems , 2000 .

[42]  E. Vaadia,et al.  Preparatory activity in motor cortex reflects learning of local visuomotor skills , 2003, Nature Neuroscience.

[43]  Karl J. Friston,et al.  A free energy principle for the brain , 2006, Journal of Physiology-Paris.

[44]  R. Bellman,et al.  Dynamic Programming and Markov Processes , 1960 .

[45]  Rodney A. Brooks,et al.  Elephants don't play chess , 1990, Robotics Auton. Syst..

[46]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[47]  Peter Bakker,et al.  Robot see, robot do: An overview of robot imitation , 1996 .

[48]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[49]  Fred Rothganger,et al.  Model-Free Learning and Control in a Mobile Robot , 2009, 2009 Fifth International Conference on Natural Computation.

[50]  Yiannis Demiris,et al.  Perceiving the unusual: Temporal properties of hierarchical motor representations for action perception , 2006, Neural Networks.

[51]  Chrystopher L. Nehaniv,et al.  Action, State and Effect Metrics for Robot Imitation , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[52]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[53]  K. Kaplan.  H. Haken, Synergetics. An Introduction. Nonequilibrium Phase Transitions and Self-Organization in Physics, Chemistry, and Biology (2nd Edition). XI + 355 S., 152 Abb. Berlin—Heidelberg—New York 1978. Springer-Verlag. DM 66,00 , 1980 .

[54]  Yiannis Demiris,et al.  Distributed, predictive perception of actions: a biologically inspired robotics architecture for imitation and learning , 2003, Connect. Sci..

[55]  R. Dillmann,et al.  TEACHING SERVICE ROBOTS COMPLEX TASKS : PROGRAMMING BY DEMONSTRATION FOR WORKSHOP AND HOUSEHOLD ENVIRONMENTS , 2001 .

[56]  D. Wolpert,et al.  Motor prediction , 2001, Current Biology.

[57]  G. Rizzolatti,et al.  A unifying view of the basis of social cognition , 2004, Trends in Cognitive Sciences.

[58]  Fabien Lagriffoul,et al.  Activity Recognition Using an Egocentric Perspective of Everyday Objects , 2007, UIC.

[59]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[60]  Maja J. Mataric,et al.  Automated Derivation of Primitives for Movement Classification , 2000, Auton. Robots.

[61]  Gillian M. Hayes,et al.  A Robot Controller Using Learning by Imitation , 1994 .

[62]  Christopher G. Atkeson,et al.  Learning from observation using primitives , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[63]  H. Haken.  Synergetics: an Introduction, Nonequilibrium Phase Transitions and Self-organization in Physics, Chemistry, and Biology , 1977 .

[64]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[65]  C. Sparrow.  The Fractal Geometry of Nature , 1984 .

[66]  Paul E. Utgoff,et al.  Shift of bias for inductive concept learning , 1984 .

[67]  Robin R. Murphy,et al.  Introduction to AI Robotics , 2000 .

[68]  Rüdiger Dillmann,et al.  Building elementary robot skills from human demonstration , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[69]  L. E. Berk,et al.  Scaffolding Children's Learning: Vygotsky and Early Childhood Education. NAEYC Research into Practice Series. Volume 7. , 1995 .

[70]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[71]  Rolf Pfeifer,et al.  Sensory - motor coordination: The metaphor and beyond , 1997, Robotics Auton. Syst..

[72]  K. Dautenhahn,et al.  Imitation in Animals and Artifacts , 2002 .

[73]  Monica N. Nicolescu,et al.  Natural methods for robot task learning: instructive demonstrations, generalization and practice , 2003, AAMAS '03.

[74]  Brandon R. Rohrer.  BECCA: A Brain Emulating Cognition and Control Architecture. , 2008 .

[75]  Aude Billard,et al.  Active Teaching in Robot Programming by Demonstration , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[76]  W. Ashby,et al.  Principles of the self-organizing dynamic system. , 1947, The Journal of general psychology.

[77]  J. Mazziotta,et al.  Cortical mechanisms of human imitation. , 1999, Science.

[78]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[79]  Jonathan Baxter,et al.  Learning internal representations , 1995, COLT '95.

[80]  Dana H. Ballard,et al.  Recognizing teleoperated manipulations , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[81]  Brandon Rohrer.  S-Learning: A Model-Free, Case-Based Algorithm for Robot Learning and Control , 2009, ICCBR.

[82]  Karl J. Friston.  Learning and inference in the brain , 2003, Neural Networks.

[83]  C. Keysers,et al.  Towards a unifying neural theory of social cognition. , 2006, Progress in brain research.

[84]  M. Matarić,et al.  Behavior-Based Segmentation of Demonstrated Tasks , 2006 .

[85]  M. Arbib,et al.  Language within our grasp , 1998, Trends in Neurosciences.

[86]  F. A. Mussa-lvaldi,et al.  Convergent force fields organized in the frog's spinal cord , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[87]  B. Bertenthal,et al.  Does Perception of Biological Motion Rely on Specific Brain Regions? , 2001, NeuroImage.

[88]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[89]  Jay Hegdé,et al.  Reappraising the Functional Implications of the Primate Visual Anatomical Hierarchy , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[90]  Daniel M. Wolpert,et al.  Hierarchical MOSAIC for movement generation , 2003 .

[91]  Thomas Hellström,et al.  A formalism for learning from demonstration , 2010, Paladyn J. Behav. Robotics.

[92]  Stefan Schaal,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[93]  Maja J. Mataric,et al.  Designing and Understanding Adaptive Group Behavior , 1995, Adapt. Behav..

[94]  Monica N. Nicolescu,et al.  Experience-based representation construction: learning from human and robot teachers , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[95]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[96]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[97]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[98]  Scott T. Grafton,et al.  Premotor Cortex Activation during Observation and Naming of Familiar Tools , 1997, NeuroImage.

[99]  Rolf Pfeifer,et al.  Understanding intelligence , 2020, Inequality by Design.

[100]  C. Gross.  Brain, Vision, Memory: Tales in the History of Neuroscience , 1998 .

[101]  Howard Hunt Pattee,et al.  Hierarchy Theory: The Challenge of Complex Systems , 1973 .

[102]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[103]  Thomas Hellström,et al.  Real-time path planning using a simulator-inthe-loop , 2009 .

[104]  Aude Billard,et al.  Reinforcement learning for imitating constrained reaching movements , 2007, Adv. Robotics.

[105]  D. George,et al.  A hierarchical Bayesian model of invariant pattern recognition in the visual cortex , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[106]  D. Perrett,et al.  Visual neurones responsive to faces in the monkey temporal cortex , 2004, Experimental Brain Research.

[107]  Paul E. Utgoff,et al.  On integrating apprentice learning and reinforcement learning , 1996 .

[108]  Stuart A. Kauffman,et al.  ORIGINS OF ORDER , 2019, Origins of Order.

[109]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[110]  Richard Alan Peters,et al.  Robonaut task learning through teleoperation , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[111]  S. Ochs.  Integrative Activity of the Brain: An Interdisciplinary Approach , 1968 .

[112]  Larry A. Rendell,et al.  Layered Concept-Learning and Dynamically Variable Bias Management , 1987, IJCAI.

[113]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[114]  V. Gallese.  The manifold nature of interpersonal relations: the quest for a common mechanism. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[115]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[116]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[117]  K. Doya,et al.  A unifying computational framework for motor control and social interaction. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[118]  Tomaso Poggio,et al.  Generalization in vision and motor control , 2004, Nature.

[119]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[120]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[121]  Nathan Delson,et al.  Robot programming by human demonstration: the use of human inconsistency in improving 3D robot trajectories , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[122]  Dileep George,et al.  How the brain might work: a hierarchical and temporal model for learning and recognition , 2008 .

[123]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[124]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[125]  Christian Balkenius,et al.  Generalization and Specialization in Reinforcement Learning , 2007 .

[126]  Chrystopher L. Nehaniv,et al.  Imitation with ALICE: learning to imitate corresponding actions across dissimilar embodiments , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[127]  M. Kawato,et al.  A hierarchical neural-network model for control and learning of voluntary movement , 2004, Biological Cybernetics.

[128]  Mohammed Yeasin,et al.  Toward automatic robot programming: learning human skill from visual data , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[129]  Werner,et al.  Complexity in natural landform patterns , 1999, Science.

[130]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[131]  Karl J. Friston.  The free-energy principle: a rough guide to the brain? , 2009, Trends in Cognitive Sciences.

[132]  Kazuhito Yokoi,et al.  Recognition and Generation of Leg Primitive Motions for Dance Imitation by a Humanoid Robot , 2003 .

[133]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[134]  Thomas Hellström,et al.  Behavior recognition for segmentation of demonstrated tasks , 2008 .

[135]  Juergen Schmidhuber,et al.  A General Method For Incremental Self-Improvement And Multi-Agent Learning In Unrestricted Environme , 1999 .

[136]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[137]  Y. Ho,et al.  Simple Explanation of the No-Free-Lunch Theorem and Its Implications , 2002 .