An adaptive background biased depth map hole-filling method for Kinect

The launch of Kinect provides a convenient way to access the depth information in real time. However the depth map quality still needs to be enhanced for 3D visual applications. In this paper, an adaptive background biased depth map hole-filling method is proposed. First, depth holes caused by abnormal reflection are filled by color similarity in-painting, and a soft decision for color similarity checking is performed by the use of probabilities in random walks color segmentation. Afterwards it is assumed that the lost information in the rest of depth holes belongs to the background. The background depth information is extracted by automatic thresholding in the neighborhood of each hole. Depth holes are in-painted with the background information in their local neighborhood. Combination of color similarity in-painting and background biased in-painting is able to perform depth map hole-filling adaptively for different kinds of depth holes for Kinect. The hole-filling results and virtual view synthesis results show that the Kinect depth map quality can be improved significantly by the proposed method.

[1]  Na-Eun Yang,et al.  Depth hole filling using the depth distribution of neighboring regions of depth holes in the Kinect sensor , 2012, 2012 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2012).

[2]  W. J. Woodall,et al.  Using the microsoft kinect for 3D map building and teleoperation , 2012, Proceedings of the 2012 IEEE/ION Position, Location and Navigation Symposium.

[3]  Shipeng Li,et al.  Texture-assisted Kinect depth inpainting , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[4]  Daniel Herrera C,et al.  Joint depth and color camera calibration with distortion correction. , 2012, IEEE transactions on pattern analysis and machine intelligence.

[5]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[6]  Yo-Sung Ho,et al.  Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3-D video , 2009, 2009 Picture Coding Symposium.

[7]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[8]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[9]  Toshiaki Fujii,et al.  View generation with 3D warping using depth information for FTV , 2009, Signal Process. Image Commun..

[10]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[11]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[12]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[13]  Lai-Man Po,et al.  Watershed and Random Walks based depth estimation for semi-automatic 2D to 3D image conversion , 2012, 2012 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2012).

[14]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[15]  Yo-Sung Ho,et al.  Depth Map Up-Sampling Using Random Walk , 2011, PSIVT.

[16]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.