Robotic grasping: from wrench space heuristics to deep learning policies

Abstract The robotic grasping task persists as a modern industry problem that seeks autonomous, fast implementation, and efficient techniques. Domestic robots are also a reality demanding a delicate and accurate human–machine interaction, with precise robotic grasping and handling. From decades ago, with analytical heuristics, to recent days, with the new deep learning policies, grasping in complex scenarios is still the aim of several works’ that propose distinctive approaches. In this context, this paper aims to cover recent methodologies’ development and discuss them, showing state-of-the-art challenges and the gap to industrial applications deployment. Given the complexity of the related issue associated with the elaborated proposed methods, this paper formulates some fair and transparent definitions for results’ assessment to provide researchers with a clear and standardised idea of the comparison between the new proposals.

[1]  Marie Jonsson,et al.  Automated material handling in composite manufacturing using pick-and-place systems – a review , 2018, Robotics and Computer-Integrated Manufacturing.

[2]  Kate Saenko,et al.  Grasp Pose Detection in Point Clouds , 2017, Int. J. Robotics Res..

[3]  Matei T. Ciocarlie,et al.  Hand Posture Subspaces for Dexterous Robotic Grasping , 2009, Int. J. Robotics Res..

[4]  L. Ingber.  Very fast simulated re-annealing , 1989 .

[5]  M. Roa,et al.  Finding locally optimum force-closure grasps , 2009 .

[6]  Luís F. Rocha,et al.  AdaptPack studio translator: translating offline programming to real palletizing robots , 2020, Ind. Robot.

[7]  Tamim Asfour,et al.  Planning grasps for robotic hands using a novel object representation based on the medial axis transform , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Zhouping Yin,et al.  Active curved surface deforming of flexible conformal electronics by multi-fingered actuator , 2020, Robotics Comput. Integr. Manuf..

[9]  Panfeng Huang,et al.  Convolutional multi-grasp detection using grasp path for RGBD images , 2019, Robotics Auton. Syst..

[10]  Van-Duc Nguyen,et al.  Constructing stable grasps in 3D , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[11]  Dan Ding,et al.  A complete and efficient algorithm for searching 3-D form-closure grasps in the discrete domain , 2004, IEEE Transactions on Robotics.

[12]  Antonio Bicchi,et al.  On the Closure Properties of Robotic Grasping , 1995, Int. J. Robotics Res..

[13]  Kenneth Y. Goldberg,et al.  Learning Deep Policies for Robot Bin Picking by Simulating Robust Grasping Sequences , 2017, CoRL.

[14]  José Boaventura-Cunha,et al.  Reconfigurable Grasp Planning Pipeline with Grasp Synthesis and Selection Applied to Picking Operations in Aerospace Factories , 2021, Robotics Comput. Integr. Manuf..

[15]  Brenna Argall,et al.  Grasp detection for assistive robotic manipulation , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Alberto Rodriguez,et al.  Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Ashutosh Saxena,et al.  Efficient grasping from RGBD images: Learning using a new rectangle representation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[18]  Yunhui Liu,et al.  Qualitative test and force optimization of 3-D frictional form-closure grasps using linear programming , 1998, IEEE Trans. Robotics Autom..

[19]  Franco Failli,et al.  Planning grasps for industrial robotized applications using neural networks , 2000 .

[20]  Máximo A. Roa,et al.  Grasp quality measures: review and performance , 2014, Autonomous Robots.

[21]  Henrik I. Christensen,et al.  Automatic grasp planning using shape primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[22]  Roderic A. Grupen,et al.  Learning prospective pick and place behavior , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[23]  Van-Duc Nguyen,et al.  Constructing force-closure grasps in 3D , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[24]  Peter K. Allen,et al.  An SVM learning approach to robotic grasping , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[25]  Mian Muhammad Awais,et al.  Learning from demonstration in robots: Experimental comparison of neural architectures , 2011 .

[26]  Christopher Kanan,et al.  Robotic grasp detection using deep convolutional neural networks , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Kuan-Ting Yu,et al.  Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching , 2019, The International Journal of Robotics Research.

[28]  António Paulo Moreira,et al.  Assessment of Robotic Picking Operations Using a 6 Axis Force/Torque Sensor , 2016, IEEE Robotics and Automation Letters.

[29]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[30]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[31]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[32]  Kate Saenko,et al.  Learning a visuomotor controller for real world robotic grasping using simulated depth images , 2017, CoRL.

[33]  Robert Platt,et al.  Using Geometry to Detect Grasp Poses in 3D Point Clouds , 2015, ISRR.

[34]  U. Castiello.  The neuroscience of grasping , 2005, Nature Reviews Neuroscience.

[35]  Di Guo,et al.  A hybrid deep architecture for robotic grasp detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[36]  A. Paulo Moreira,et al.  Stereo-based real-time 6-DoF work tool tracking for robot programing by demonstration , 2016 .

[37]  Panfeng Huang,et al.  Detecting Graspable Rectangles of Objects in Robotic Grasping , 2020 .

[38]  Jürgen Leitner,et al.  Learning robust, real-time, reactive robotic grasping , 2019, Int. J. Robotics Res..

[39]  Abhinav Gupta,et al.  Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[40]  Dan Ding,et al.  Computing 3-D optimal form-closure grasps , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[41]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[42]  Joseph Redmon,et al.  Real-time grasp detection using convolutional neural networks , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[43]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[44]  Philippe Gorce,et al.  A multistage neural network architecture to learn hand grasping posture , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[46]  B. Dizioglu,et al.  Mechanics of form closure , 1984 .

[47]  Luís F. Rocha,et al.  AdaptPack Studio: an automated intelligent framework for offline factory programming , 2020, Ind. Robot.

[48]  Robert Platt.  Learning grasp strategies composed of contact relative motions , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[49]  Marco Santello,et al.  Patterns of Hand Motion during Grasping and the Influence of Sensory Guidance , 2002, The Journal of Neuroscience.

[50]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[51]  Asokan Thondiyath,et al.  Optimizing force closure grasps on 3D objects using a modified genetic algorithm , 2018, Soft Comput..

[52]  L. Jakobson,et al.  A neurological dissociation between perceiving objects and grasping them , 1991, Nature.

[53]  Yunhui Liu.  Computing n-Finger Form-Closure Grasps on Polygonal Objects , 2000, Int. J. Robotics Res..

[54]  Erhan Oztop,et al.  A biologically inspired learning to grasp system , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[55]  Patricio A. Vela,et al.  Real-World Multiobject, Multigrasp Detection , 2018, IEEE Robotics and Automation Letters.

[56]  Peter K. Allen,et al.  Grasp Planning via Decomposition Trees , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[57]  A K Bejczy,et al.  Sensors, Controls, and Man-Machine Interface for Advanced Teleoperation , 1980, Science.

[58]  Tamim Asfour,et al.  Integrated Grasp Planning and Visual Object Localization For a Humanoid Robot with Five-Fingered Hands , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[59]  Carlo Alberto Avizzano,et al.  A study on picking objects in cluttered environments: Exploiting depth features for a custom low-cost universal jamming gripper , 2020, Robotics Comput. Integr. Manuf..

[60]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[61]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.

[62]  Anis Sahbani,et al.  On computing robust n-finger force-closure grasps of 3D objects , 2009, 2009 IEEE International Conference on Robotics and Automation.

[63]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.

[64]  Andrew J. Davison,et al.  Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task , 2017, CoRL.

[65]  Lionel Birglen,et al.  A statistical review of industrial robotic grippers , 2018 .

[66]  Mathieu Aubry,et al.  Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[67]  Hong Liu,et al.  On computing three-finger force-closure grasps of 2-D and 3-D objects , 2003, IEEE Trans. Robotics Autom..

[68]  Jean Ponce,et al.  On computing three-finger force-closure grasps of polygonal objects , 1991, Fifth International Conference on Advanced Robotics 'Robots in Unstructured Environments.

[69]  Danica Kragic,et al.  Large-scale supervised learning of the grasp robustness of surface patch pairs , 2016, 2016 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR).

[70]  P. Dasgupta,et al.  The future of robotics , 2017, Investigative and Clinical Urology.

[71]  Jianwei Zhang,et al.  Learning to grasp everyday objects using reinforcement-learning with automatic value cut-off , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[72]  Nassir Navab,et al.  Dealing with Ambiguity in Robotic Grasping via Multiple Predictions , 2018, ACCV.

[73]  Maciej Klimek,et al.  Improved GQ-CNN: Deep Learning Model for Planning Robust Grasps , 2018, ArXiv.

[74]  Brahim Chaib-draa,et al.  Sparse Dictionary Learning for Identifying Grasp Locations , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[75]  Brahim Chaib-draa,et al.  GQ-STN: Optimizing One-Shot Grasp Detection based on Robustness Classifier , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[76]  Stefano Caselli,et al.  A 3D shape segmentation approach for robot grasping by parts , 2012, Robotics Auton. Syst..

[77]  Xinyu Li,et al.  A novel robotic grasp detection method based on region proposal networks , 2020, Robotics Comput. Integr. Manuf..

[78]  Ken Goldberg,et al.  Learning ambidextrous robot grasping policies , 2019, Science Robotics.

[79]  Daniela Rus,et al.  Learning Object Grasping for Soft Robot Hands , 2018, IEEE Robotics and Automation Letters.

[80]  Xinyu Liu,et al.  Dex-Net 3.0: Computing Robust Robot Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning , 2017, ArXiv.

[81]  Vincent Babin,et al.  Stable and repeatable grasping of flat objects on hard surfaces using passive and epicyclic mechanisms , 2019, Robotics and Computer-Integrated Manufacturing.