Simultaneous human-robot adaptation for effective skill transfer

In this paper, we propose and implement a human-in-the loop robot skill synthesis framework that involves simultaneous adaptation of the human and the robot. In this framework, the human demonstrator learns to control the robot in real-time to make it perform a given task. At the same time, the robot learns from the human guided control creating a non-trivial coupled dynamical system. The research question we address is how this system can be tuned to facilitate faster skill transfer or improve the performance level of the transferred skill. In the current paper we report our initial work for the latter. At the beginning of the skill transfer session, the human demonstrator controls the robot exclusively as in teleoperation. As the task performance improves the robot takes increasingly more share in control, eventually reaching full autonomy. The proposed framework is implemented and shown to work on a physical cart-pole setup. To assess whether simultaneous learning has advantage over the standard sequential learning (where the robot learns from the human observation but does not interfere with the control) experiments with two groups of subjects were performed. The results indicate that the final autonomous controller obtained via simultaneous learning has a higher performance measured as the average deviation from the upright posture of the pole.

[1]  Stefan Schaal,et al.  Learning Control in Robotics , 2010, IEEE Robotics & Automation Magazine.

[2]  Yu Sun,et al.  Determining the benefit of human input in human-in-the-loop robotic systems , 2013, 2013 IEEE RO-MAN.

[3]  Kerstin Dautenhahn,et al.  Self-Imitation and Environmental Scaffolding for Robot Teaching , 2007 .

[4]  Tadej Petric,et al.  Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach , 2014, Auton. Robots.

[5]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[6]  Chrystopher L. Nehaniv,et al.  Teaching robots by moulding behavior and scaffolding the environment , 2006, HRI '06.

[7]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[8]  Gordon Cheng,et al.  Dexterous Skills Transfer by Extending Human Body Schema to a Robotic Hand , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[9]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[10]  Joshua G. Hale,et al.  Human sensorimotor learning for humanoid robot skill synthesis , 2011, Adapt. Behav..

[11]  Joshua G. Hale,et al.  Using Humanoid Robots to Study Human Behavior , 2000, IEEE Intell. Syst..

[12]  Magnus Egerstedt,et al.  Human-in-the-Loop: Terminal constraint receding horizon control with human inputs , 2010, 2010 IEEE International Conference on Robotics and Automation.

[13]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[14]  Stefan Schaal,et al.  The New Robotics—towards Human-centered Machines , 2007 .

[15]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[16]  Brian Moore,et al.  Robotic grasping and manipulation through human visuomotor learning , 2012, Robotics Auton. Syst..

[17]  Luka Peternel,et al.  Learning of compliant human–robot interaction using full-body haptic interface , 2013, Adv. Robotics.

[18]  W. Marsden I and J , 2012 .

[19]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[20]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[21]  Stefan Schaal,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[22]  Ieee Robotics,et al.  IEEE robotics & automation magazine , 1994 .

[23]  Jan Peters,et al.  Reinforcement Learning to Adjust Robot Movements to New Situations , 2010, IJCAI.

[24]  Stefan Schaal,et al.  Learning to Control in Operational Space , 2008, Int. J. Robotics Res..

[25]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[26]  Satoru Goto,et al.  Human direct teaching of industrial articulated robot arms based on force-free control , 2006, Artificial Life and Robotics.

[27]  Gordon Cheng,et al.  Extensive Human Training for Robot Skill Synthesis: Validation on a Robotic Hand , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[28]  Luka Peternel,et al.  Humanoid robot posture-control learning in real-time based on human sensorimotor learning ability , 2013, 2013 IEEE International Conference on Robotics and Automation.

[29]  Jan Peters,et al.  Reinforcement Learning to Adjust Robot Movements to New Situations , 2010, IJCAI.

[30]  Jun Morimoto,et al.  Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning , 2000, Robotics Auton. Syst..

[31]  Magnus Egerstedt,et al.  Human-in-the-loop: MPC for shared control of a quadruped rescue robot , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Jan Peters,et al.  Nonamemanuscript No. (will be inserted by the editor) Reinforcement Learning to Adjust Parametrized Motor Primitives to , 2011 .

[33]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .