Force-based End-to-end Training of a Mobile Manipulator: ―Learning Human Following from Applied Force―

In this paper, we present a novel method for inducing a mobile robotic manipulator through human force using an end-to-end approach. We applied deep reinforcement learning to train our robot to estimate human intention using the error and the error rate of change for each joint in the manipulator as input state. We used reward method based on stimulus using the displacement between the manipulator’s joint positions when no load is received and the current joint positions. The larger the displacement is the larger the stimulus which gives a negative reward. We successfully trained the robot to respond appropriately to an external force.

[1]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[2]  Keng Peng Tee,et al.  A Framework of Human–Robot Coordination Based on Game Theory and Policy Iteration , 2016, IEEE Transactions on Robotics.

[3]  Frank L. Lewis,et al.  Optimized Assistive Human–Robot Interaction Using Reinforcement Learning , 2016, IEEE Transactions on Cybernetics.

[4]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[5]  Takashi Minato,et al.  Physical Human-Robot Interaction: Mutual Learning and Adaptation , 2012, IEEE Robotics & Automation Magazine.

[6]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[7]  Kazuhiro Kosuge,et al.  Dance partner robot - Ms DanceR , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[8]  F A GELDARD,et al.  Some neglected possibilities of communication. , 1960, Science.

[9]  Heni Ben Amor,et al.  A system for learning continuous human-robot interactions from human-human demonstrations , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Kazuo Tanie,et al.  Assist system for carrying a long object with a human-analysis of a human cooperative behavior in the vertical direction , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).