Benchmarking In-Hand Manipulation

The purpose of this benchmark is to evaluate the planning and control aspects of robotic in-hand manipulation systems. The goal is to assess the system's ability to change the pose of a hand-held object by either using the fingers, environment or a combination of both. Given an object surface mesh from the YCB data-set, we provide examples of initial and goal states (i.e. static object poses and fingertip locations) for various in-hand manipulation tasks. We further propose metrics that measure the error in reaching the goal state from a specific initial state, which, when aggregated across all tasks, also serves as a measure of the system's in-hand manipulation capability. We provide supporting software, task examples, and evaluation results associated with the benchmark.

[1]  Zhaopeng Chen,et al.  Towards a functional evaluation of manipulation performance in dexterous robotic hand design , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Matthew T. Mason,et al.  Fast Planning for 3D Any-Pose-Reorienting Using Pivoting , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Mariapaola D'Imperio,et al.  A dexterous gripper for in-hand manipulation , 2016, 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[4]  M. Levas.  OBBTree : A Hierarchical Structure for Rapid Interference Detection , .

[5]  Aaron M. Dollar,et al.  Dexterous manipulation with underactuated elastic hands , 2011, 2011 IEEE International Conference on Robotics and Automation.

[6]  Emanuel Todorov,et al.  Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system , 2018, 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR).

[7]  Nikhil Chavan Dafle,et al.  Pneumatic shape-shifting fingers to reorient and grasp , 2018, 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE).

[8]  Aaron M. Dollar,et al.  Variable-Friction Finger Surfaces to Enable Within-Hand Manipulation via Gripping and Sliding , 2018, IEEE Robotics and Automation Letters.

[9]  Aaron M. Dollar,et al.  The GR2 Gripper: An Underactuated Hand for Open-Loop In-Hand Planar Manipulation , 2016, IEEE Transactions on Robotics.

[10]  Zoran Popovic,et al.  Discovery of complex behaviors through contact-invariant optimization , 2012, ACM Trans. Graph..

[11]  Alberto Rodriguez,et al.  In-Hand Manipulation via Motion Cones , 2018, Robotics: Science and Systems.

[12]  Jian Shi,et al.  Dynamic In-Hand Sliding Manipulation , 2017, IEEE Trans. Robotics.

[13]  Stefan Ulbrich,et al.  The OpenGRASP benchmarking suite: An environment for the comparative analysis of grasping and dexterous manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Jeffrey C. Trinkle,et al.  Dextrous manipulation by rolling and finger gaiting , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[15]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[16]  S. Shankar Sastry,et al.  Grasping and Coordinated Manipulation by a Multifingered Robot Hand , 1989, Int. J. Robotics Res..

[17]  Aaron M. Dollar,et al.  Vision-based model predictive control for within-hand precision manipulation with underactuated grippers , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Masatoshi Ishikawa,et al.  Dynamic regrasping using a high-speed multifingered hand and a high-speed vision system , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[19]  Kostas E. Bekris,et al.  Path Planning for Within-Hand Manipulation over Learned Representations of Safe States , 2018, ISER.

[20]  Danica Kragic,et al.  Dexterous Manipulation Graphs , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Jan Peters,et al.  Regularizing Reinforcement Learning with State Abstraction , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Siddhartha S. Srinivasa,et al.  Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set , 2015, IEEE Robotics & Automation Magazine.

[23]  Aaron M. Dollar,et al.  A two-fingered robot gripper with large object reorientation range , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Sergey Levine,et al.  Optimal control with learned local models: Application to dexterous manipulation , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Aaron M. Dollar,et al.  Robust Precision Manipulation With Simple Process Models Using Visual Servoing Techniques With Disturbance Rejection , 2019, IEEE Transactions on Automation Science and Engineering.

[26]  OpenAI.  Learning Dexterous In-Hand Manipulation. , 2018 .

[27]  Silvia Cruciani,et al.  In-hand manipulation using three-stages open loop pivoting , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Keenan Crane,et al.  The heat method for distance computation , 2017, Commun. ACM.

[29]  Tucker Hermans,et al.  Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulation , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Masatoshi Ishikawa,et al.  Rubik's Cube Handling Using a High-Speed Multi-Fingered Hand and a High-Speed Vision System , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Aaron M. Dollar,et al.  Learning Modes of Within-Hand Manipulation , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Sergey Levine,et al.  Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations , 2017, Robotics: Science and Systems.

[33]  Nikhil Chavan Dafle,et al.  Stable Prehensile Pushing: In-Hand Manipulation with Alternating Sticking Contacts , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Tucker Hermans,et al.  Relaxed-Rigidity Constraints: In-Grasp Manipulation using Purely Kinematic Trajectory Optimization , 2017, Robotics: Science and Systems.

[35]  Jakub W. Pachocki,et al.  Learning dexterous in-hand manipulation , 2018, Int. J. Robotics Res..

[36]  Aaron M. Dollar,et al.  Learning task-specific models for dexterous, in-hand manipulation with simple, adaptive robot hands , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[37]  Helge J. Ritter,et al.  Integrating vision, haptics and proprioception into a feedback controller for in-hand manipulation of unknown objects , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Siddhartha S. Srinivasa,et al.  Extrinsic dexterity: In-hand manipulation with external forces , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Peter K. Allen,et al.  Compliant manipulation with a dextrous robot hand , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[40]  Daniela Rus,et al.  In-Hand Dexterous Manipulation of Piecewise-Smooth 3-D Objects , 1999, Int. J. Robotics Res..

[41]  Tucker Hermans,et al.  Relaxed-rigidity constraints: kinematic trajectory optimization and collision avoidance for in-grasp manipulation , 2018, Autonomous Robots.

[42]  Kenji Tahara,et al.  Stable pinching by controlling finger relative orientation of robotic fingers with rolling soft tips , 2018, Robotica.

[43]  Dinesh Manocha,et al.  Fast distance queries with rectangular swept sphere volumes , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[44]  Yiannis Karayiannidis,et al.  Dexterous manipulation with compliant grasps and external contacts , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[45]  Daniela Rus.  Coordinated Manipulation of Objects in a Plane , 1997, Algorithmica.

[46]  Or Tslil,et al.  Robotic Swing-Up Regrasping Manipulation Based on the Impulse–Momentum Approach and cLQR Control , 2016, IEEE Transactions on Robotics.

[47]  Daniela Rus,et al.  Autonomous Object Manipulation Using a Soft Planar Grasping Manipulator , 2015, Soft robotics.

[48]  Danica Kragic,et al.  Reinforcement Learning for Pivoting Task , 2017, ArXiv.

[49]  Du Q. Huynh,et al.  Metrics for 3D Rotations: Comparison and Analysis , 2009, Journal of Mathematical Imaging and Vision.

[50]  Jimmy A. Jørgensen,et al.  VisGraB: A benchmark for vision-based grasping , 2012, Paladyn J. Behav. Robotics.

[51]  Danica Kragic,et al.  Adaptive control for pivoting with visual and tactile feedback , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[52]  Jan Peters,et al.  Learning robot in-hand manipulation with tactile features , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).