Depth map Super-Resolution based on joint dictionary learning

Although Time-of-Flight (ToF) camera can provide real-time depth information from a real scene, the resolution of depth map captured by ToF camera is rather limited compared to HD color cameras, and thus it cannot be directly used in 3D reconstruction. In order to handle this problem, this paper proposes a novel compressive sensing (CS) and dictionary learning based depth map super-resolution (SR) method, which transforms a low resolution depth map to a high resolution depth map. Different from previous depth map SR methods, this algorithm uses a joint dictionary learning method with both low and high resolution depth maps, and this method also builds a sparse vector classification method which is used in depth map SR. Experimental results show that the proposed method outperforms state-of-the-art methods for depth map super-resolution.

[1]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[2]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[3]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[4]  Paul Tseng,et al.  A coordinate gradient descent method for nonsmooth separable minimization , 2008, Math. Program..

[5]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, ACM Trans. Graph..

[7]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  E.J. Candes Compressive Sampling , 2022 .

[9]  Xuelong Li,et al.  Single-image super-resolution via local learning , 2011, Int. J. Mach. Learn. Cybern..

[10]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[12]  Qi Tian,et al.  Image Annotation by Input–Output Structural Grouping Sparsity , 2012, IEEE Transactions on Image Processing.

[13]  Yueting Zhuang,et al.  Cross-media semantic representation via bi-directional learning to rank , 2013, ACM Multimedia.

[14]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[15]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[17]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[18]  Yi Yang,et al.  Image Attribute Adaptation , 2014, IEEE Transactions on Multimedia.

[19]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[20]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[21]  Zhou Yu,et al.  Sparse Multi-Modal Hashing , 2014, IEEE Transactions on Multimedia.

[22]  Xuelong Li,et al.  Joint Learning for Single-Image Super-Resolution via a Coupled Constraint , 2012, IEEE Transactions on Image Processing.

[23]  Yihong Gong,et al.  Resolution enhancement based on learning the sparse association of image patches , 2010, Pattern Recognit. Lett..