Region-based depth recovery for highly sparse depth maps

The accurate recovery of missing values in depth maps is an important problem in computer vision and image processing. In depth maps with large, irregular missing regions (i.e., sparse depth maps) inaccuracies arise when depth values of known pixels are used to recover depth near object edges and depth discontinuities (leakage). In order to overcome this problem, we propose an iterative region-based depth recovery method. In the proposed approach, the depth recovery problem is solved iteratively for each region of the segmented image in order to reduce the effect of leakage. Quantitative and qualitative experiments conducted on real data sets show promising results when comparing the proposed approach with state-of-the-art methods.

[1]  Jilin Liu,et al.  Guided depth enhancement via a fast marching method , 2013, Image Vis. Comput..

[2]  Bart Selman,et al.  Unstructured human activity detection from RGBD images , 2011, 2012 IEEE International Conference on Robotics and Automation.

[3]  Jitendra Malik,et al.  Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Michael S. Brown,et al.  High quality depth map upsampling for 3D-TOF cameras , 2011, 2011 International Conference on Computer Vision.

[5]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[6]  Sebastian Thrun,et al.  An Application of Markov Random Fields to Range Sensing , 2005, NIPS.

[7]  Seung-Won Jung,et al.  Enhancement of Image and Depth Map Using Adaptive Joint Trilateral Filter , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Michael Beetz,et al.  Inpainting of Missing Values in the Kinect Sensor's Depth Maps Based on Background Estimates , 2014, IEEE Sensors Journal.

[9]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[10]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Shijian Lu,et al.  Discriminative Multi-modal Feature Fusion for RGBD Indoor Scene Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Reinhard Koch,et al.  Time‐of‐Flight Cameras in Computer Graphics , 2010, Comput. Graph. Forum.

[13]  Ruigang Yang,et al.  Spatial-Temporal Fusion for High Accuracy Depth Maps Using Dynamic MRFs , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Yao Wang,et al.  Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model , 2014, IEEE Transactions on Image Processing.

[15]  Haibo Wang,et al.  Kinect Depth Inpainting via Graph Laplacian with TV21 Regularization , 2013, 2013 2nd IAPR Asian Conference on Pattern Recognition.

[16]  Shih-Fu Chang,et al.  Interactive Segmentation on RGBD Images via Cue Selection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Horst Bischof,et al.  Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Gamini Dissanayake,et al.  A robust RGB-D SLAM algorithm , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Minh N. Do,et al.  Depth Video Enhancement Based on Weighted Mode Filtering , 2012, IEEE Transactions on Image Processing.

[20]  Xiaojin Gong,et al.  Guided Depth Enhancement via Anisotropic Diffusion , 2013, PCM.

[21]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[22]  Juho Kannala,et al.  Depth Map Inpainting under a Second-Order Smoothness Prior , 2013, SCIA.

[23]  Peter Kulchyski and , 2015 .

[24]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[26]  Sanja Fidler,et al.  Holistic Scene Understanding for 3D Object Detection with RGBD Cameras , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[28]  Stephen Lin,et al.  Object-Based Multiple Foreground Segmentation in RGBD Video , 2017, IEEE Transactions on Image Processing.

[29]  Bevilacqua Marco,et al.  Visibility estimation and joint inpainting of lidar depth maps , 2016 .