Depth image super-resolution reconstruction based on a modified joint trilateral filter

Depth image super-resolution (SR) is a technique that uses signal processing technology to enhance the resolution of a low-resolution (LR) depth image. Generally, external database or high-resolution (HR) images are needed to acquire prior information for SR reconstruction. To overcome the limitations, a depth image SR method without reference to any external images is proposed. In this paper, a high-quality edge map is first constructed using a sparse coding method, which uses a dictionary learned from the original images at different scales. Then, the high-quality edge map is used to guide the interpolation for depth images by a modified joint trilateral filter. During the interpolation, some information of gradient and structural similarity (SSIM) are added to preserve the detailed information and suppress the noise. The proposed method can not only preserve the sharpness of image edge, but also avoid the dependence on database. Experimental results show that the proposed method is superior to some state-of-the-art depth image SR methods.

[1]  Gang Xiong,et al.  The application of the depth camera in the social manufacturing: A review , 2016, 2016 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI).

[2]  Horst Bischof,et al.  Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Xueying Qin,et al.  Deep Depth Super-Resolution: Learning Depth Super-Resolution Using Deep Convolutional Neural Network , 2016, ACCV.

[4]  Tae-Kyun Kim,et al.  Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[6]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[7]  Horst Bischof,et al.  A Deep Primal-Dual Network for Guided Depth Super-Resolution , 2016, BMVC.

[8]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[9]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[10]  Xuezhi Xiang,et al.  A modified joint trilateral filter based depth map refinement method , 2016, 2016 12th World Congress on Intelligent Control and Automation (WCICA).

[11]  Yao Zhao,et al.  Single depth image super-resolution with multiple residual dictionary learning and refinement , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[12]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[13]  Yehoshua Y. Zeevi,et al.  Image enhancement and denoising by complex diffusion processes , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Hongbin Zha,et al.  Similarity-Aware Patchwork Assembly for Depth Image Super-resolution , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Antonio Fernández-Caballero,et al.  Mobile robot map building from time-of-flight camera , 2012, Expert Syst. Appl..

[16]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Rogério Schmidt Feris,et al.  Edge guided single depth image super resolution , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[18]  Tae-Kyun Kim,et al.  Real-Time Articulated Hand Pose Estimation Using Semi-supervised Transductive Regression Forests , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Andrew W. Fitzgibbon,et al.  Efficient regression of general-activity human poses from depth images , 2011, 2011 International Conference on Computer Vision.

[21]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[22]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[23]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[25]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[26]  Gabriel J. Brostow,et al.  Patch Based Synthesis for Single Depth Image Super-Resolution , 2012, ECCV.

[27]  Long Ye,et al.  A Modified Joint Trilateral Filter for Depth Image Super Resolution , 2016, IFTC.

[28]  Kai-Lung Hua,et al.  Edge-Preserving Depth Map Upsampling by Joint Trilateral Filter , 2018, IEEE Transactions on Cybernetics.