Non-essentiality of correlation between image and depth map in free viewpoint image coding: Accurate depth map case

We show non-essentiality of using a correlation between an image and a depth map for depth-image-based rendering (DIBR), when we use an accurate depth map. For coding of DIBR, an edge preserving filter, which jointly use the image and the depth map, as a post filter is a suitable approach. The joint filter not only removes coding distortion, but also improves accuracy of the coded depth map itself. Considering the development 3D technology, the accuracy of the input depth map will be improved. If we use an accurate depth map, e.g. the ground truth, the accuracy improvement of the joint filter becomes little. To reveal the fact, we use various codecs (JPEG, JPEG2000, and H.264/AVC) and use two state-of-the-arts filters, which are the post filter set as the non-joint filter, and the weighed mode filter as the joint filter. Experimental results show that the post filters do not require the joint image, and self-sustained types of the non-joint filter have better performance.

[1]  Patrick Hébert,et al.  Median Filtering in Constant Time , 2007, IEEE Transactions on Image Processing.

[2]  Masayuki Tanimoto,et al.  Comparison between blur transfer and blur re-generation in depth image based rendering , 2014, 2014 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[3]  Lai-Man Po,et al.  Depth map misalignment correction and dilation for DIBR view synthesis , 2013, Signal Process. Image Commun..

[4]  Minh N. Do,et al.  Depth Video Enhancement Based on Weighted Mode Filtering , 2012, IEEE Transactions on Image Processing.

[5]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Toshiaki Fujii,et al.  View generation with 3D warping using depth information for FTV , 2009, Signal Process. Image Commun..

[7]  Takeshi Naemura,et al.  Graph Cut Based Continuous Stereo Matching Using Locally Shared Labels , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Yutaka Ishibashi,et al.  Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement , 2013, VISAPP.

[9]  Thomas Wiegand,et al.  3-D Video Representation Using Depth Maps , 2011, Proceedings of the IEEE.

[10]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[11]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[12]  Yutaka Ishibashi,et al.  Filter based alpha matting for depth image based rendering , 2013, 2013 Visual Communications and Image Processing (VCIP).

[13]  Richard Szeliski,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[14]  C. Lawrence Zitnick,et al.  High-quality video view interpolation , 2006, SIGGRAPH Courses.

[15]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Yutaka Ishibashi,et al.  Removing depth map coding distortion by using post filter set , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[17]  Dong Tian,et al.  New Depth Coding Techniques With Utilization of Corresponding Video , 2011, IEEE Transactions on Broadcasting.

[18]  Yo-Sung Ho,et al.  Depth Coding Using a Boundary Reconstruction Filter for 3-D Video Systems , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).