6D Pose Estimation of Occlusion-Free Objects for Robotic Bin-Picking Using PPF-MEAM With 2D Images (Occlusion-Free PPF-MEAM)

Pose estimation that locates objects in a bin is necessary for a robotic bin picking system. Although many algorithms have shown high performance in pose estimation, most algorithms estimate the poses of objects regardless of their occlusion. This can reduce the success rate in picking up the object. To resolve this issue, we propose a novel pipeline that estimates a pose only for occlusion-free objects based on point pair feature-based pose estimation with multiple edge appearance model (PPF-MEAM). The proposed method detects occlusion-free objects in the 2D image captured by a camera with a convolutional neural network framework. Next, corresponding point clouds of occlusion-free objects need to be extracted by using their locations in the 2D image. we propose a robust extraction method that finds the 3D points corresponding to image pixels in the 2D image to reduce the effect of the calibration errors between the camera and 3D sensor. The point cloud of the occlusion-free objects is finally input into a pipeline of PPF-MEAM to estimate the pose of the object. The experiment results prove that the proposed method is about 50% faster 30% higher in terms of pose estimation success rate compared with the original PPF. Moreover, it increases the success rate of picking tasks compared with the original PPF-MEAM.

[1]  Koichi Hashimoto,et al.  Feedback projection for 3D measurements under complex lighting conditions , 2017, 2017 American Control Conference (ACC).

[2]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Markus Vincze,et al.  OUR-CVFH - Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram for Object Recognition and 6DOF Pose Estimation , 2012, DAGM/OAGM Symposium.

[4]  Zhang Yong,et al.  High-efficiency and high-accuracy digital image correlation for three-dimensional measurement , 2015 .

[5]  Nico Blodow,et al.  Aligning point cloud views using persistent feature histograms , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Song Zhang,et al.  Superfast phase-shifting method for 3-D shape measurement. , 2010, Optics express.

[8]  Koichi Hashimoto,et al.  Model-based virtual visual servoing with point cloud data , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Nassir Navab,et al.  Model globally, match locally: Efficient and robust 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[13]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[14]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Koichi Hashimoto,et al.  Fast Prediction of a Worker’s Reaching Motion Without a Skeleton Model (F-PREMO) , 2020, IEEE Access.

[16]  Yisheng Guan,et al.  A 3D object detection and pose estimation pipeline using RGB-D images , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[17]  Federico Tombari,et al.  SHOT: Unique signatures of histograms for surface and texture description , 2014, Comput. Vis. Image Underst..

[18]  Vincent Lepetit,et al.  Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes , 2011, 2011 International Conference on Computer Vision.

[19]  Tinne Tuytelaars,et al.  Discriminatively Trained Templates for 3D Object Detection: A Real Time Scalable Approach , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  Vincent Lepetit,et al.  Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes , 2012, ACCV.

[21]  Dieter Fox,et al.  Unsupervised Feature Learning for RGB-D Based Object Recognition , 2012, ISER.

[22]  Mingyu Li,et al.  Curve Set Feature-Based Robust and Fast Pose Estimation Algorithm , 2017, Sensors.

[23]  Jun Kinugawa,et al.  A projection-based making-human-feel-safe system for human-robot cooperation , 2016, 2016 IEEE International Conference on Mechatronics and Automation.

[24]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Shogo Arai,et al.  A Convolutional Neural Network for Point Cloud Instance Segmentation in Cluttered Scene Trained by Synthetic Data Without Color , 2020, IEEE Access.

[26]  Nico Blodow,et al.  CAD-model recognition and 6DOF pose estimation using 3D cues , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[27]  Bolei Zhou,et al.  SegICP: Integrated deep semantic segmentation and pose estimation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[30]  Michael Greenspan,et al.  Generalized 4-Points Congruent Sets for 3D Registration , 2014, 2014 2nd International Conference on 3D Vision.

[31]  Jun Kinugawa,et al.  2D Object Localization Based Point Pair Feature for Pose Estimation , 2018, 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[32]  Jun Kinugawa,et al.  Deep-Learning based Robust Edge Detection for Point Pair Feature-based Pose Estimation with Multiple Edge Appearance Models , 2019, 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[33]  Jun Kinugawa,et al.  Adaptive Task Scheduling for an Assembly Task Coworker Robot Based on Incremental Learning of Human's Motion Patterns , 2017, IEEE Robotics and Automation Letters.

[34]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Federico Tombari,et al.  A performance evaluation of point pair features , 2018, Comput. Vis. Image Underst..

[36]  Jun Kinugawa,et al.  Point Pair Feature-Based Pose Estimation with Multiple Edge Appearance Models (PPF-MEAM) for Robotic Bin Picking , 2018, Sensors.

[37]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[38]  Song Zhang.  Recent progresses on real-time 3D shape measurement using digital fringe projection techniques , 2010 .

[39]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[40]  Ming-Yu Liu,et al.  Learning to Rank 3D Features , 2014, ECCV.

[41]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  S Arai,et al.  Fast sensor scheduling with communication costs for sensor networks , 2010, Proceedings of the 2010 American Control Conference.

[43]  Koichi Hashimoto,et al.  FAst Detection Algorithm for 3D Keypoints (FADA-3K) , 2020, IEEE Access.

[44]  Koichi Hashimoto,et al.  Fast Sensor Scheduling for Spatially Distributed Sensors , 2011, IEEE Transactions on Automatic Control.

[45]  Jin Xu,et al.  3D Object Recognition and Pose Estimation From Point Cloud Using Stably Observed Point Pair Feature , 2020, IEEE Access.

[46]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[48]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Dieter Fox,et al.  PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes , 2017, Robotics: Science and Systems.

[50]  Dah-Jye Lee,et al.  Review of stereo vision algorithms and their suitability for resource-limited systems , 2013, Journal of Real-Time Image Processing.

[51]  Kensuke Harada,et al.  Iterative Visual Recognition for Learning Based Randomized Bin-Picking , 2016, ISER.

[52]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.