Investigating Deep Reinforcement Learning For Grasping Objects With An Anthropomorphic Hand

Grasping objects with high dimensional controllers such as an anthropomorphic hand using reinforcement learning is a challenging problem. In this work we experiment with a 16-D simulated version of a prosthetic hand developed for SouthHampton Hand Assessment Procedure (SHAP). We demonstrate that it is possible to learn successful grasp policies for an anthropomorphic hand from scratch using deep reinforcement learning. We find that our grasping model is robust to sensor noise, variations in object shape, position of the object and physical parameters such as the density of the object. Under these variations, we also investigate the utility of touch sensing for grasping objects. We believe that our results and analysis provide useful insights and strong baselines for future research into the exciting direction of object manipulation with anthropomorphic hands using proprioceptive and other sensory feedback.

[1]  Robert D. Howe,et al.  A compliant, underactuated hand for robust manipulation , 2013, Int. J. Robotics Res..

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Kate Saenko,et al.  High precision grasp pose detection in dense clutter , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  C. Light,et al.  Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: normative data, reliability, and validity. , 2002, Archives of physical medicine and rehabilitation.

[5]  Sergey Levine,et al.  Learning dexterous manipulation for a soft robotic hand from human demonstrations , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Robert D. Howe,et al.  Simple, Robust Autonomous Grasping in Unstructured Environments , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[7]  Vikash Kumar,et al.  Fast, strong and compliant pneumatic actuation for dexterous tendon-driven hands , 2013, 2013 IEEE International Conference on Robotics and Automation.

[8]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[9]  Emanuel Todorov,et al.  The UW Hand : A Low-cost , 20-DOF Tendon-driven Hand with Fast and Compliant Actuation , 2012 .

[10]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Martin A. Riedmiller,et al.  Acquiring visual servoing reaching and grasping skills using neural reinforcement learning , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[12]  Sergey Levine,et al.  Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations , 2017, Robotics: Science and Systems.

[13]  Abhinav Gupta,et al.  Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Alexander Herzog,et al.  Learning of grasp selection based on shape-templates , 2014, Auton. Robots.

[15]  Helge J. Ritter,et al.  Platform portable anthropomorphic grasping with the bielefeld 20-DOF shadow and 9-DOF TUM hand , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[17]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[18]  Jeannette Bohg,et al.  Leveraging big data for grasp planning , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[20]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[21]  Matei T. Ciocarlie,et al.  The Columbia grasp database , 2009, 2009 IEEE International Conference on Robotics and Automation.

[22]  Oliver Brock,et al.  Automated co-design of soft hand morphology and control strategy for grasping , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  D. Rus,et al.  Design, fabrication and control of soft robots , 2015, Nature.

[24]  Alberto Rodriguez,et al.  From caging to grasping , 2011, Int. J. Robotics Res..

[25]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.

[26]  Oliver Brock,et al.  A novel type of compliant and underactuated robotic hand for dexterous grasping , 2016, Int. J. Robotics Res..

[27]  Sergey Levine,et al.  Learning Dexterous Manipulation Policies from Experience and Imitation , 2016, ArXiv.