Efficient online adaptation with stochastic recurrent neural networks

Autonomous robots need to interact with unknown and unstructured environments. For continuous online adaptation in lifelong learning scenarios, they need sample-efficient mechanisms to adapt to changing environments, constraints, tasks and capabilities. In this paper, we introduce a framework for online motion planning and adaptation based on a bio-inspired stochastic recurrent neural network. By using the intrinsic motivation signal cognitive dissonance with a mental replay strategy, the robot can learn from few physical interactions and can therefore adapt to novel environments in seconds. We evaluate our online planning and adaptation framework on a KUKA LWR arm. The efficient online adaptation is shown by learning unknown workspace constraints sample-efficient within few seconds while following given via points.

[1]  L. Festinger.  Cognitive dissonance. , 1962, Scientific American.

[2]  J. Kagan.  Motives and development. , 1972, Journal of personality and social psychology.

[3]  Sebastian Thrun,et al.  Lifelong robot learning , 1993, Robotics Auton. Syst..

[4]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[5]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation , 2000 .

[6]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[7]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[8]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[9]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[10]  Juyang Weng,et al.  Developmental Robotics: Theory and Experiments , 2004, Int. J. Humanoid Robotics.

[11]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[12]  Andrew G. Barto,et al.  Intrinsically Motivated Reinforcement Learning: A Promising Framework for Developmental Robot Learning , 2005 .

[13]  A.G. Alleyne,et al.  A survey of iterative learning control , 2006, IEEE Control Systems.

[14]  Jürgen Schmidhuber,et al.  Optimal Artificial Curiosity, Creativity, Music, and the Fine Arts , 2005 .

[15]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[16]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[17]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[18]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[19]  Andrew G. Barto,et al.  Competence progress intrinsic motivation , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[20]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[21]  Stephen Hart,et al.  Learning Generalizable Control Programs , 2011, IEEE Transactions on Autonomous Mental Development.

[22]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[23]  Genzhong Wu,et al.  Adaptive Iterative Learning Control for Robot Manipulators , 2012 .

[24]  David Johan Christensen,et al.  A distributed and morphology-independent strategy for adaptive locomotion in self-reconfigurable modular robots , 2013, Robotics Auton. Syst..

[25]  Marco Mirolli,et al.  Intrinsically Motivated Learning Systems: An Overview , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[26]  Frank Kirchner,et al.  Incremental learning of skill collections based on intrinsic motivation , 2013, Front. Neurorobot..

[27]  V. Santucci.  Intrinsic motivation signals for driving the acquisition of multiple tasks : A simulated robotic study , 2013 .

[28]  T. Martin McGinnity,et al.  Novelty Detection as an Intrinsic Motivation for Cumulative Learning Robots , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[29]  Vincent Padois,et al.  Emergence of humanoid walking behaviors from mixed-integer model predictive control , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[31]  Jan Peters,et al.  Recurrent Spiking Networks Solve Planning Tasks , 2016, Scientific Reports.

[32]  Jan Peters,et al.  Deep spiking networks for model-based planning in humanoids , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).