A Grasping CNN with Image Segmentation for Mobile Manipulating Robot

This paper presents a grasping convolutional neural network with image segmentation for mobile manipulating robot. The proposed method is cascaded by a feature pyramid network FPN and a grasping network DrGNet. The FPN network combined with point cloud clustering is used to obtain the mask of the target object. Then, the grayscale map and the depth map corresponding to the target object are combined and sent to the DrGNet network for providing multi-scale images. On this basis, depthwise separable convolution is used for encoding. The results of encoders are refined according to the light-weight RefineNet as well as sSE, which can achieve a better grasp detection. The proposed method is verified by the experiments on mobile manipulating robot.

[1]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Peter Corke,et al.  Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach , 2018, Robotics: Science and Systems.

[3]  Alexey A. Shvets,et al.  Feature Pyramid Network for Multi-class Land Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Ian D. Reid,et al.  Light-Weight RefineNet for Real-Time Semantic Segmentation , 2018, BMVC.

[5]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[6]  Jianbin Tang,et al.  GraspNet: An Efficient Convolutional Neural Network for Real-time Grasp Detection for Low-powered Devices , 2018, IJCAI.

[7]  Christopher Kanan,et al.  Robotic grasp detection using deep convolutional neural networks , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Oliver Kroemer,et al.  Learning robot grasping from 3-D images with Markov Random Fields , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[11]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[12]  E. AmirM.Ghalamzan,et al.  Safe robotic grasping: Minimum impact-force grasp selection , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[15]  Antonio Morales,et al.  Vision-based three-finger grasp synthesis constrained by hand geometry , 2006, Robotics Auton. Syst..

[16]  Joseph Redmon,et al.  Real-time grasp detection using convolutional neural networks , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Daniela Rus,et al.  Learning Object Grasping for Soft Robot Hands , 2018, IEEE Robotics and Automation Letters.

[19]  Nassir Navab,et al.  Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks , 2018, MICCAI.

[20]  Yu Wang,et al.  Underwater Bioinspired Propulsion: From Inspection to Manipulation , 2020, IEEE Transactions on Industrial Electronics.

[21]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.