Transfer of knowledge for a climbing Virtual Human: A reinforcement learning approach

In the reinforcement learning literature, transfer is the capability to reuse on a new problem what has been learnt from previous experiences on similar problems. Adapting transfer properties for robotics is a useful challenge because it can reduce the time spent in the first exploration phase on a new problem. In this paper we present a transfer framework adapted to the case of a climbing Virtual Human (VH). We show that our VH learns faster to climb a wall after having learnt on a different previous wall.

[1]  M. Stolle,et al.  Knowledge Transfer Using Local Features , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[2]  Aude Billard,et al.  Goal-Directed Imitation in a Humanoid Robot , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[3]  G. Konidaris.  A Framework for Transfer in Reinforcement Learning , 2006 .

[4]  Sethu Vijayakumar,et al.  Adaptive Optimal Control for Redundantly Actuated Arms , 2008, SAB.

[5]  Tomoaki Yano,et al.  Development of a self-contained wall climbing robot with scanning type suction cups , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[6]  W. T. Dempster,et al.  Properties of body segments based on size and weight , 1967 .

[7]  Bernhard Schölkopf,et al.  Learning Inverse Dynamics: a Comparison , 2008, ESANN.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Timothy Bretl,et al.  Free-Climbing with a Multi-Use Robot , 2006, ISER.

[10]  Ernest P Hanavan,et al.  A mathematical model of the human body , 1964 .

[11]  Alain Micaelli,et al.  Dynamic balance control of humanoids for multiple grasps and non coplanar frictional contacts , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[12]  Timothy Bretl,et al.  Multi-Step Motion Planning for Free-Climbing Robots , 2004, WAFR.

[13]  Shigeo Hirose,et al.  Walking and running of the quadruped wall-climbing robot , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[14]  Ronald S. Fearing,et al.  Synthetic gecko foot-hair micro/nano-structures for future wall-climbing robots , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[15]  C. Karen Liu,et al.  Synthesis of complex dynamic character motion from simple animations , 2002, ACM Trans. Graph..

[16]  Axel van Lamsweerde,et al.  Learning machine learning , 1991 .

[17]  Stefan Schaal,et al.  Reinforcement Learning for Operational Space Control , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[18]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[19]  Andrew G. Barto,et al.  Autonomous shaping: knowledge transfer in reinforcement learning , 2006, ICML.

[20]  Richard S. Sutton,et al.  Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming , 1990, ML.

[21]  Ernest P. Hanavan.  A Personalized Mathematical Model of the Human Body , 1966 .

[22]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .