Robotic grasp detection based on image processing and random forest

Real-time grasp detection plays a key role in manipulation, and it is also a complex task, especially for detecting how to grasp novel objects. This paper proposes a very quick and accurate approach to detect robotic grasps. The main idea is to perform grasping of novel objects in a typical RGB-D scene view. Our goal is not to find the best grasp for every object but to obtain the local optimal grasps in candidate grasp rectangles. There are three main contributions to our detection work. Firstly, an improved graph segmentation approach is used to do objects detection and it can separate objects from the background directly and fast. Secondly, we develop a morphological image processing method to generate candidate grasp rectangles set which avoids us to search grasp rectangles globally. Finally, we train a random forest model to predict grasps and achieve an accuracy of 94.26%. The model is mainly used to score every element in our candidate grasps set and the one gets the highest score will be converted to the final grasp configuration for robots. For real-world experiments, we set up our system on a tabletop scene with multiple objects and when implementing robotic grasps, we control Baxter robot with a different inverse kinematics strategy rather than the built-in one.

[1]  Richard P. Paul,et al.  Robot manipulators : mathematics, programming, and control : the computer control of robot manipulators , 1981 .

[2]  Ariel Shamir,et al.  Inverse Kinematics Techniques in Computer Graphics: A Survey , 2018, Comput. Graph. Forum.

[3]  Quoc V. Le,et al.  Grasping novel objects with depth segmentation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Hong Liu,et al.  A new algorithm for three-finger force-closure grasp of polygonal objects , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[5]  F. Frances Yao,et al.  Computational Geometry , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[6]  Kostas Daniilidis,et al.  Hand-Eye Calibration Using Dual Quaternions , 1999, Int. J. Robotics Res..

[7]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.

[8]  Xiaogang Wang,et al.  Crafting GBD-Net for Object Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Markus Vincze,et al.  Shape Based Learning for Grasping Novel Objects in Cluttered Scenes , 2012, SyRoCo.

[10]  Nanning Zheng,et al.  RoI-based Robotic Grasp Detection in Object Overlapping Scenes Using Convolutional Neural Network , 2018, ArXiv.

[11]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[12]  Peter Corke,et al.  Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach , 2018, Robotics: Science and Systems.

[13]  Miao Li,et al.  Grasping Novel Objects with Real-Time Obstacle Avoidance , 2018, ICSR.

[14]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[15]  Markus Vincze,et al.  Learning grasps with topographic features , 2015, Int. J. Robotics Res..

[16]  Robert Platt,et al.  Using Geometry to Detect Grasp Poses in 3D Point Clouds , 2015, ISRR.

[17]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[18]  Calyampudi R. Rao,et al.  Generalized inverse of a matrix and its applications , 1972 .

[19]  Markus Vincze,et al.  Empty the basket - a shape based learning approach for grasping piles of unknown objects , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Dan Ding,et al.  Computation of fingertip positions for a form-closure grasp , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[21]  Patricio A. Vela,et al.  Deep Grasp: Detection and Localization of Grasps with Deep Neural Networks , 2018, ArXiv.

[22]  Kate Saenko,et al.  High precision grasp pose detection in dense clutter , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Robert Platt,et al.  Localizing Handle-Like Grasp Affordances in 3D Point Clouds , 2014, ISER.

[25]  A. A. Maciejewski,et al.  Obstacle Avoidance , 2005 .

[26]  Di Guo,et al.  Deep vision networks for real-time robotic grasp detection , 2016 .

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[29]  Ashutosh Saxena,et al.  Efficient grasping from RGBD images: Learning using a new rectangle representation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[30]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[31]  Brahim Chaib-draa,et al.  Convolutional Residual Network for Grasp Localization , 2017, 2017 14th Conference on Computer and Robot Vision (CRV).

[32]  Henrik I. Christensen,et al.  Automatic grasp planning using shape primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[33]  Kate Saenko,et al.  Grasp Pose Detection in Point Clouds , 2017, Int. J. Robotics Res..

[34]  Ana Consuelo Huaman.  Grasp selection strategies for robot manipulation using a superquadric-based object representation , 2016 .

[35]  Joseph Redmon,et al.  Real-time grasp detection using convolutional neural networks , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[37]  James J. Kuffner,et al.  OpenRAVE: A Planning Architecture for Autonomous Robotics , 2008 .

[38]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Christopher Kanan,et al.  Robotic grasp detection using deep convolutional neural networks , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[40]  Federico Thomas,et al.  Grasping Unknown Objects in Clutter by Superquadric Representation , 2017, 2018 Second IEEE International Conference on Robotic Computing (IRC).