Guided Depth Enhancement via Anisotropic Diffusion

In this paper, we propose to conduct inpainting and upsampling for defective depth maps when aligned color images are given. These tasks are referred to as the guided depth enhancement problem. We formulate the problem based on the heat diffusion framework. The pixels with known depth values are treated as the heat sources and the depth enhancement is performed via diffusing the depth from these sources to unknown regions. The diffusion conductivity is designed in terms of the guidance color image so that a linear anisotropic diffusion problem is formed. We further cast the steady state problem of this diffusion into the famous random walk model, by which the enhancement is achieved efficiently by solving a sparse linear system. The proposed algorithm is quantitatively evaluated on the Middlebury stereo dataset and is applied to inpaint Kinect data and upsample Lidar's range data. Comparisons to the commonly used bilateral filter and Markov Random Field based methods are also presented, showing that our algorithm is competent.

[1]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Yong Huang,et al.  Texture decomposition by harmonics extraction from higher order statistics , 2004, IEEE Trans. Image Process..

[3]  Michael S. Brown,et al.  High quality depth map upsampling for 3D-TOF cameras , 2011, 2011 International Conference on Computer Vision.

[4]  Sebastian Thrun,et al.  An Application of Markov Random Fields to Range Sensing , 2005, NIPS.

[5]  Martin Rumpf,et al.  Anisotropic geometric diffusion in surface processing , 2000 .

[6]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[8]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Danny Barash,et al.  Bilateral Filtering and Anisotropic Diffusion: Towards a Unified Viewpoint , 2001, Scale-Space.

[10]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, International Journal of Computer Vision.

[11]  Jonathan T. Barron,et al.  A category-level 3-D object dataset: Putting the Kinect to work , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[12]  Takeo Kanade,et al.  Distributed cosegmentation via submodular optimization on anisotropic diffusion , 2011, 2011 International Conference on Computer Vision.

[13]  Jakob Wasza,et al.  Real-time preprocessing for dense 3-D range imaging on the GPU: Defect interpolation, bilateral temporal averaging and guided filtering , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[14]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Ruigang Yang,et al.  Spatial-Temporal Fusion for High Accuracy Depth Maps Using Dynamic MRFs , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Xiaojin Gong,et al.  Guided inpainting and filtering for Kinect depth maps , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[17]  John J. Leonard,et al.  Cooperative AUV Navigation Using a Single Surface Craft , 2009, FSR.

[18]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[19]  Stig Larsson,et al.  Partial differential equations with numerical methods , 2003, Texts in applied mathematics.

[20]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[21]  Thomas Deselaers,et al.  ClassCut for Unsupervised Class Segmentation , 2010, ECCV.

[22]  William Whittaker,et al.  Camera and LIDAR Fusion for Mapping of Actively Illuminated Subterranean Voids , 2009, FSR.

[23]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[24]  Sebastian Thrun,et al.  Upsampling range data in dynamic environments , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Mongi A. Abidi,et al.  Spatial and Temporal Enhancement of Depth Images Captured by a Time-of-Flight Depth Sensor , 2010, 2010 20th International Conference on Pattern Recognition.

[26]  Guangming Shi,et al.  Structure guided fusion for depth map inpainting , 2013, Pattern Recognit. Lett..

[27]  Jianfei Cai,et al.  A diffusion approach to seeded image segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Christopher Joseph Pal,et al.  On Learning Conditional Random Fields for Stereo , 2010, International Journal of Computer Vision.

[29]  R. Deriche,et al.  Anisotropic Diffusion Partial Differential Equations for Multichannel Image Regularization: Framework and Applications , 2007 .

[30]  Jilin Liu,et al.  Guided depth enhancement via a fast marching method , 2013, Image Vis. Comput..

[31]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.