Depth map recovery based on optimal scheduling of invalid point filling

This paper proposes a recovery method which is commendably able to fill the invalid points in the depth image obtained with Kinectv2. Two primary tasks of the invalid points filling process are filling priority evaluation and optimal filling value prediction. At first, we put forward an entropy-based measurement for the filling priority. Then a metric of entropy that combines Valid Point Number and Neighboring Standard Deviation is proposed to sort all the invalid points. Secondly, the optimal prediction of the filling value is calculated by finding the valid neighbor that is most similar to the average of its neighborhood. Finally, the proposed method is applied to each invalid point to complete the recovery of the depth image. Experimental results show that our method can effectively fill the invalid points in the depth image and performs better than some competitive algorithms in accuracy and robustness.

[1]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[2]  Didier Stricker,et al.  Comparison of Kinect V1 and V2 Depth Images in Terms of Accuracy and Precision , 2016, ACCV Workshops.

[3]  Luis Salgado,et al.  Efficient spatio-temporal hole filling strategy for Kinect depth maps , 2012, Electronic Imaging.

[4]  Ruimin Hu,et al.  Color image guided locality regularized representation for Kinect depth holes filling , 2013, 2013 Visual Communications and Image Processing (VCIP).

[5]  Ruimin Hu,et al.  Filling Kinect depth holes via position-guided matrix completion , 2016, Neurocomputing.

[6]  Na-Eun Yang,et al.  Depth hole filling using the depth distribution of neighboring regions of depth holes in the Kinect sensor , 2012, 2012 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2012).

[7]  Senchun Chai,et al.  Depth Filtering in 3D Reconstruction of Indoor Scenes Based on Kinect , 2014, 2014 Seventh International Symposium on Computational Intelligence and Design.

[8]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  艾而帝,et al.  Microsoft Kinect 虛擬復健系統設計 , 2013 .

[10]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[11]  Irek Defée,et al.  Performance of similarity measures based on histograms of local image feature vectors , 2007, Pattern Recognit. Lett..

[12]  Seung-Won Jung,et al.  Directional Joint Bilateral Filter for Depth Images , 2014, Sensors.

[13]  Dmitriy Vatolin,et al.  Temporal filtering for depth maps generated by Kinect depth camera , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).