Reinforcement learning of shared control for dexterous telemanipulation: Application to a page turning skill

The ultimate goal of this study is to develop a method that can accomplish dexterous manipulation of various non-rigid objects by a robotic hand. In this paper, we propose a novel model-free approach using reinforcement learning to learn a shared control policy for dexterous telemanipulation by a human operator. A shared control policy is a probabilistic mapping from the human operator's (master) action and complementary sensor data to the robot (slave) control input for robot actuators. Through the learning process, our method can optimize the shared control policy so that it cooperates to the operator's policy and compensates the lack of sensory information of the operator using complementary sensor data to enhance the dexterity. To validate our method, we adopted a page turning task by telemanipulation and developed an experimental platform with a paper page model and a robot fingertip in simulation. Since the human operator cannot perceive the tactile information of the robot, it may not be as easy as humans do directly. Experimental results suggest that our method is able to learn task-relevant shared control for flexible and enhanced dexterous manipulation by a teleoperated robotic fingertip without tactile feedback to the operator.

[1]  Farbod Fahimi,et al.  Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[2]  Tomohiro Shibata,et al.  Fast Reinforcement Learning for Three-Dimensional Kinetic Human–Robot Cooperation with an EMG-to-Activation Model , 2011, Adv. Robotics.

[3]  Mark R. Cutkosky,et al.  Feedback Strategies for Telemanipulation with Shared Control of Object Handling Forces , 2005, Presence: Teleoperators & Virtual Environments.

[4]  Jan Peters,et al.  Policy Search for Motor Primitives in Robotics , 2008, NIPS 2008.

[5]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[6]  Masashi Sugiyama,et al.  Reward-Weighted Regression with Sample Reuse for Direct Policy Search in Reinforcement Learning , 2011, Neural Computation.

[7]  Jian Huang,et al.  Study of reinforcement learning based shared control of walking-aid robot , 2013, Proceedings of the 2013 IEEE/SICE International Symposium on System Integration.

[8]  Silvestro Micera,et al.  On the Shared Control of an EMG-Controlled Prosthetic Hand: Analysis of User–Prosthesis Interaction , 2008, IEEE Transactions on Robotics.

[9]  Takamitsu Matsubara,et al.  Reinforcement learning of a motor skill for wearing a T-shirt using topology coordinates , 2013, Adv. Robotics.

[10]  Tsuneo Yoshikawa,et al.  Acquisition of a page turning skill for a multifingered hand using reinforcement learning , 2004, Adv. Robotics.

[11]  Stefano Carpin,et al.  Combining imitation and reinforcement learning to fold deformable planar objects , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Mark A. Minor,et al.  Hybrid Force–Velocity Sliding Mode Control of a Prosthetic Hand , 2008, IEEE Transactions on Biomedical Engineering.

[13]  Tomohiro Shibata,et al.  Assist-as-needed robotic trainer based on reinforcement learning and its application to dart-throwing , 2014, Neural Networks.