Image assisted upsampling of depth map via nonlocal similarity

The depth resolution of TOF cameras is poor and the resulting depth maps are noisy. Therefore, it is highly desirable to develop powerful image processing tools to enhance the resolution and suppress the noise of depth maps. In this paper, we propose a new image-assisted upsampling method for depth map. A spatially adaptive iterative singular-value thresholding (SAIST) with image-guided patch clustering strategy is developed and compared with previous image-guided depth map upsampling techniques. Overall, the proposed scheme is capable of better preserving salient global structure information. Extensive experimental results are reported to justify the superiority of the proposed method.

[1]  Sebastian Thrun,et al.  A Noise‐aware Filter for Real‐time Depth Upsampling , 2008 .

[2]  Michael S. Brown,et al.  High quality depth map upsampling for 3D-TOF cameras , 2011, 2011 International Conference on Computer Vision.

[3]  Sebastian Thrun,et al.  An Application of Markov Random Fields to Range Sensing , 2005, NIPS.

[4]  Oscar C. Au,et al.  Depth map denoising using graph-based transform and group sparsity , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[5]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[6]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[7]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[8]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[9]  Horst Bischof,et al.  Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  D. Louis Collins,et al.  Non-local MRI upsampling , 2010, Medical Image Anal..

[11]  Martin Vetterli,et al.  Adaptive wavelet thresholding for image denoising and compression , 2000, IEEE Trans. Image Process..

[12]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[13]  Antonio Torralba,et al.  Properties and applications of shape recipes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..