Single Depth Map Super-resolution with Local Self-similarity

Consumer depth sensors such as time-of-flight camera or Kinect have gained significant popularity in recently. However, the captured depth maps suffer from limited spatial resolution and a variety of noise, making such depth maps difficult to be directly applied in related applications. In this paper, we present a novel single depth map super-resolution method, aiming to reconstruct high-resolution depth map from its associated low-resolution depth map without any auxiliary information. Particularly, we exploit the depth local self-similarity to assist in constructing patch pairs in terms of high-resolution and low-resolution depth edge patches, and then deduce a high-resolution depth edge map via Markov model. Finally, we implement a joint bilateral filter to reconstruct the high-resolution depth map. Experimental results show that our method overcomes existing methods on the benchmark database as well as Kinect captured depth maps.

[1]  Gabriel J. Brostow,et al.  Patch Based Synthesis for Single Depth Image Super-Resolution , 2012, ECCV.

[2]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[3]  Aljoscha Smolic,et al.  3D video and free viewpoint video - From capture to display , 2011, Pattern Recognit..

[4]  S. Foix,et al.  Lock-in Time-of-Flight (ToF) Cameras: A Survey , 2011, IEEE Sensors Journal.

[5]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[8]  Horst Bischof,et al.  ATGV-Net: Accurate Depth Super-Resolution , 2016, ECCV.

[9]  Rogério Schmidt Feris,et al.  Edge guided single depth image super resolution , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[10]  Michael S. Brown,et al.  High quality depth map upsampling for 3D-TOF cameras , 2011, 2011 International Conference on Computer Vision.

[11]  Sebastian Thrun,et al.  An Application of Markov Random Fields to Range Sensing , 2005, NIPS.

[12]  Hongbin Zha,et al.  Similarity-Aware Patchwork Assembly for Depth Image Super-resolution , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[14]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Horst Bischof,et al.  Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Seung-Won Jung,et al.  A Consensus-Driven Approach for Structure and Texture Aware Depth Map Upsampling , 2014, IEEE Transactions on Image Processing.

[17]  Luc Van Gool,et al.  One-Shot Person Re-identification with a Consumer Depth Camera , 2014, Person Re-Identification.

[18]  L. Rudin,et al.  Feature-oriented image enhancement using shock filters , 1990 .

[19]  Carsten Rother,et al.  Depth Super Resolution by Rigid Body Self-Similarity in 3D , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.