Residual Squeeze-and-Excitation Network with Multi-scale Spatial Pyramid Module for Fast Robotic Grasping Detection

This paper proposes an efficient, fully convolutional neural network to generate robotic grasps by using 300×300 depth images as input. Specifically, a residual squeeze-and-excitation network (RSEN) is introduced for deep feature extraction. Following the RSEN block, a multi-scale spatial pyramid module (MSSPM) is developed to obtain multi-scale contextual information. The outputs of each RSEN block and MSSPM are combined as inputs for hierarchical feature fusion. Then, the fused global features are upsampled to perform pixel-wise learning for grasping pose estimation. The experimental results on Cornell and Jacquard grasping datasets indicate that the proposed method has a fast inference speed of 5ms while achieving high grasp detection accuracy of 96.4% and 94.8% on Cornell and Jacquard, respectively, which strikes a balance between accuracy and running speed. Our method also gets a 90% physical grasp success rate with a UR5 robot arm.

[1]  Bin Li,et al.  Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset , 2020, Frontiers in Neurorobotics.

[2]  Xinyu Li,et al.  A novel robotic grasp detection method based on region proposal networks , 2020, Robotics Comput. Integr. Manuf..

[3]  Petros Maragos,et al.  Orientation Attentive Robot Grasp Synthesis , 2020, ArXiv.

[4]  Dexin Wang,et al.  SGDN: Segmentation-Based Grasp Detection Network For Unsymmetrical Three-Finger Gripper , 2020, ArXiv.

[5]  Liming Chen,et al.  Optimizing Correlated Graspability Score and Grasp Regression for Better Grasp Prediction , 2020, ArXiv.

[6]  Hironobu Fujiyoshi,et al.  Detecting layered structures of partially occluded objects for bin picking , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Ferat Sahin,et al.  Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network , 2019, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Jürgen Leitner,et al.  Learning robust, real-time, reactive robotic grasping , 2019, Int. J. Robotics Res..

[9]  Patric Jensfelt,et al.  Object Detection Approach for Robot Grasp Detection , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[10]  Shiguo Lian,et al.  Vision-based Robotic Grasping from Object Localization, Pose Estimation, Grasp Detection to Motion Planning: A Review , 2019, ArXiv.

[11]  Brahim Chaib-draa,et al.  GQ-STN: Optimizing One-Shot Grasp Detection based on Robustness Classifier , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Jie Zhao,et al.  Efficient Fully Convolution Neural Network for Generating Pixel Wise Robotic Grasps With High Resolution Images , 2019, 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[13]  Dongwon Park,et al.  Real-Time, Highly Accurate Robotic Grasp Detection using Fully Convolutional Neural Networks with High-Resolution Images , 2018, ArXiv.

[14]  Kangfu Mei,et al.  Multi-scale Residual Network for Image Super-Resolution , 2018, ECCV.

[15]  Zhiqiang Tian,et al.  ROI-based Robotic Grasp Detection for Object Overlapping Scenes , 2018, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Jianbin Tang,et al.  GraspNet: An Efficient Convolutional Neural Network for Real-time Grasp Detection for Low-powered Devices , 2018, IJCAI.

[17]  Emmanuel Dellandréa,et al.  Jacquard: A Large Scale Dataset for Robotic Grasp Detection , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Yang Zhang,et al.  Fully Convolutional Grasp Detection Network with Oriented Anchor Box , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Dongwon Park,et al.  Classification based Grasp Detection using Spatial Transformer Network , 2018, ArXiv.

[20]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[21]  Patricio A. Vela,et al.  Deep Grasp: Detection and Localization of Grasps with Deep Neural Networks , 2018, ArXiv.

[22]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Di Guo,et al.  A hybrid deep architecture for robotic grasp detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.

[25]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Christopher Kanan,et al.  Robotic grasp detection using deep convolutional neural networks , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Hong Liu,et al.  Robot grasp detection using multimodal deep convolutional neural networks , 2016 .

[28]  Stefan Leutenegger,et al.  Deep learning a grasp function for grasping under gripper pose uncertainty , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[30]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[31]  Abhinav Gupta,et al.  Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Joseph Redmon,et al.  Real-time grasp detection using convolutional neural networks , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[35]  Ashutosh Saxena,et al.  Efficient grasping from RGBD images: Learning using a new rectangle representation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[37]  Jane You,et al.  Multi-Object Grasping Detection With Hierarchical Feature Fusion , 2019, IEEE Access.

[38]  Daokui Qu,et al.  Robust Robot Grasp Detection in Multimodal Fusion , 2017 .