Object Detection for Soft Robotic Manipulation Based on RGB-D Sensors

In this paper, a visual recognition and location method using RGB-D information fusion is proposed for object detection, which is convenient for soft robotic manipulation to grasp objects. First, the environment is scanned and reconstructed by ORB-SLAM2, and the acquired color image and point cloud data are further processed, and the object feature database, which includes color and depth information, is constructed. Secondly, the point cloud in the point cloud library and the matching point cloud are put into the same coordinate system, and the ICP algorithm is used to match the point cloud to the point cloud in the point cloud library as much as possible, and the matching error is obtained, and the region of interest is obtained according to the size of the matching error. Then, if the region of interest is not uniquely determined, the object recognition is further realized by using the inception-v3 model and the transfer learning to identify the region of interest. After obtaining the only definite region, the position of the object relative to the camera is obtained through the correspondence between the color information and the point cloud data, and the object location is achieved. In order to verify the rationality of the method, a matching validation experiment was designed and a database containing multiple objects was established. In the experimental stage, the point cloud information and color information of the object are collected again from another angle, and they are matched with the point cloud in the point cloud library to get the results and analyze them. The results show that the error of the point cloud that belongs to the same object is much less than that of the point cloud that does not belong to the same object. Then the object can be identified and the location of the object can be identified successfully by color recognition.

[1]  Filip Ilievski,et al.  Soft robotics for chemists. , 2011, Angewandte Chemie.

[2]  E. Stein,et al.  Introduction to Fourier Analysis on Euclidean Spaces. , 1971 .

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yin Zhou,et al.  VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Thomas Hofmann,et al.  Support vector machine learning for interdependent and structured output spaces , 2004, ICML.

[7]  Leonidas J. Guibas,et al.  Frustum PointNets for 3D Object Detection from RGB-D Data , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  C. Keplinger,et al.  Giant voltage-induced deformation in dielectric elastomers near the verge of snap-through instability , 2013 .

[9]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[11]  Luke Fitzgerald.  Introduction to Pascal (2nd edition) , 1984 .

[12]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[13]  John A. Pawlicki,et al.  Object detection system for vehicles , 2003 .

[14]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Xuelong Li,et al.  Enhanced Biologically Inspired Model for Object Recognition , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Yang Wei-mi.  A new kind of soft robots actuating by electromagnetic force , 2013 .

[17]  Nicholas G. Polson,et al.  Data augmentation for support vector machines , 2011 .

[18]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  M. Yamakita,et al.  Active Polymers: An Overview , 2007 .

[21]  Qian ShaoMing,et al.  Pneumatic Robot Multi-fingered Dexterous Hand - ZJUT Hand , 2012 .

[22]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[23]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[24]  Andrew Zisserman,et al.  Multiple kernels for object detection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[26]  Nicholas M. Patrikalakis,et al.  An algorithm for optimal free-form object matching , 2003, Comput. Aided Des..

[27]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).