Enhancement of Image and Depth Map Using Adaptive Joint Trilateral Filter

In this paper, we present an adaptive joint trilateral filter (AJTF), which consists of domain, range, and depth filters. The AJTF is used for the joint enhancement of images and depth maps, which is achieved by suppressing the noise and sharpening the edges simultaneously. For improving the sharpness of the image and depth map, the AJTF parameters, the offsets, and the standard deviations of the range and depth filters are determined in such a way that image edges that match well with depth edges are emphasized. To this end, pattern matching between local patches in the image and depth map is performed and the matching result is utilized to adjust the AJTF parameters. Experimental results show that the AJTF produces sharpness-enhanced images and depth maps without overshoot and undershoot artifacts, while successfully reducing noise as well. A comparison of the performance of the AJTF with those of conventional image and depth enhancement algorithms shows that the proposed algorithm is effective.

[1]  Björn E. Ottersten,et al.  Pixel weighted average strategy for depth sensor data fusion , 2010, 2010 IEEE International Conference on Image Processing.

[2]  Jan P. Allebach,et al.  Adaptive Bilateral Filter for Sharpness Enhancement and Noise Removal , 2007, ICIP.

[3]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[4]  Bhabatosh Chanda,et al.  Color image compression based on block truncation coding using pattern fitting principle , 2007, Pattern Recognit..

[5]  Dong Tian,et al.  Depth map processing with iterative joint multilateral filtering , 2010, 28th Picture Coding Symposium.

[6]  Michael Elad,et al.  On the bilateral filter and ways to improve it , 2002 .

[7]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[8]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[9]  Hans-Peter Seidel,et al.  3 D Unsharp Masking for Scene Coherent Enhancement Supplemental Material 1 : Experimental Validation of the Algorithm , 2008 .

[10]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  Paolo Cignoni,et al.  A simple normal enhancement technique for interactive non-photorealistic renderings , 2005, Comput. Graph..

[12]  Erhan Ekmekcioglu,et al.  Content Adaptive Enhancement of Multi-View Depth Maps for Free Viewpoint Video , 2011, IEEE Journal of Selected Topics in Signal Processing.

[13]  Aljoscha Smolic,et al.  3DAV exploration of video-based rendering technology in MPEG , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Yo-Sung Ho,et al.  Depth Reconstruction Filter and Down/Up Sampling for Depth Coding in 3-D Video , 2009, IEEE Signal Processing Letters.

[15]  Timo Schairer,et al.  Fusion of range and color images for denoising and resolution enhancement with a non-local filter , 2010, Comput. Vis. Image Underst..

[16]  Giovanni Ramponi,et al.  Image enhancement via adaptive unsharp masking , 2000, IEEE Trans. Image Process..

[17]  Oliver Deussen,et al.  Image enhancement by unsharp masking the depth buffer , 2006, SIGGRAPH 2006.

[18]  Fatih Porikli,et al.  Constant time O(1) bilateral filtering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Jan P. Allebach,et al.  Optimal unsharp mask for image sharpening and noise removal , 2004, IS&T/SPIE Electronic Imaging.

[22]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, ECCV.

[23]  Ahmet M. Kondoz,et al.  Adaptive sharpening of depth maps for 3D-TV , 2010 .

[24]  Ahmet M. Kondoz,et al.  Edge adaptive upsampling of depth map videos for enhanced free-viewpoint video quality , 2009 .