RGB-D Camera based 3D Object Pose Estimation and Grasping

It is a great challenge to grasp 3D objects in unstructured environment. This task is closely related with object recognition, pose estimation, hand-eye calibration and grasp strategy planning. This paper focuses on the 6-DoF pose estimation and hand-eye calibration problems. Based on the point cloud provided by the RGB-D sensor, Viewpoint Feature Histogram (VFH) descriptor is used to localize the object by comparing the scene and model library. Instead of using a pan-tilt platform to build the template library, an industrial robot with in-hand camera is programmed to collect point clouds from different view angles. Distances between scene point cloud and the model point clouds are evaluated to find a group of candidate poses. The poses are further refined by aligning those point cloud pairs using Iterative Closest Point (ICP) algorithm. Although standard VFH descriptor is invariant to scale, it is sensitive to viewpoint variance, which may lead to irrational results. In order to improve the robustness, effect of the translational offset and number of pose candidates are evaluated. The hand-eye calibration process is formulated into an AX=ZB problem and solved by using quaternion rotation and least squares method. A series of experiments are performed with a RGB-D sensor and an industrial robot. The results verify that the method is effective to estimate the object poses. Considering the accuracy of the used sensor, it is proved that the proposed method has acceptable robustness and accuracy.