Dual-Way Guided Depth Image Inpainting with RGBD Image Pairs

Raw depth images acquired by low-cost range imaging sensors, such as Kinect v1 and Kinect v2, usually contain invalid regions without depth information and suffer from noise. Although many approaches are proposed to address the problems, the robustness of these approaches still needs enhancement. This paper introduces a dual-way guided inpainting method with RGBD image pairs to restore the missing depth values of invalid regions and to eliminate noise caused by acquisition apparatus. By leveraging the structural difference between the colour and depth images, the colour image is first segmented with watershed segmentation and then merged under the guidance of the simultaneously captured depth image, followed by inpainting the depth image guided by the merged colour image using Radial Basis Functions (RBFs). The proposed framework of the dual-way guided approach with the RBFs is new for depth image inpainting and outperforms the existing state-of-the-art approaches in the experimental evaluations.

[1]  Yo-Sung Ho,et al.  Depth image filter for mixed and noisy pixel removal in RGB-D camera systems , 2013, IEEE Transactions on Consumer Electronics.

[2]  Bogdan Kwolek,et al.  Human fall detection on embedded platform using depth maps and wireless accelerometer , 2014, Comput. Methods Programs Biomed..

[3]  Jilin Liu,et al.  Guided depth enhancement via a fast marching method , 2013, Image Vis. Comput..

[4]  Manolis I. A. Lourakis,et al.  T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, International Journal of Computer Vision.

[6]  Hujun Bao,et al.  Consistent Depth Maps Recovery from a Video Sequence , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[8]  Frédo Durand,et al.  Two-scale tone management for photographic look , 2006, SIGGRAPH 2006.

[9]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[10]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[11]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Hui Lin,et al.  Depth image enhancement for Kinect using region growing and bilateral filter , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[14]  Hujun Bao,et al.  Recovering consistent video depth maps via bundle optimization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Yao Wang,et al.  Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model , 2014, IEEE Transactions on Image Processing.