Error supression in view synthesis using reliability reasoning for FTV

View synthesis using depth maps is a crucial application for Free-viewpoint TV (FTV). In this paper, we propose a novel reliability based view synthesis method using two references and their depth maps. The depth estimation with stereo matching is known to be error-prone, leading to noticeable artifacts in the synthesized new views. In order to provide plausible virtual views for FTV, our focus is on the error suppression for the synthesized view. We innovatively introduce the continuous reliability using error approximation by the reference cross-check. The new view interpolation algorithm is generated with the criterion of Least Sum of Squared Errors (LSSE). Furthermore, the proposed algorithm can be considered as a reliable version of the conventional linear view blending. We experimentally demonstrate the effectiveness of our framework with MPEG standard test sequences. The results show that our method outperforms state-of-the-art view interpolation methods both at eliminating artifacts and improving PSNR.

[1]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[2]  N. Atzpadin,et al.  Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability , 2007, Signal Process. Image Commun..

[3]  Yo-Sung Ho,et al.  Boundary Filtering on Synthesized Views of 3D Video , 2008, 2008 Second International Conference on Future Generation Communication and Networking Symposia.

[4]  Masayuki Tanimoto Overview of free viewpoint television , 2006, Signal Process. Image Commun..

[5]  Masayuki Tanimoto,et al.  Multiview Imaging and 3DTV , 2007, IEEE Signal Processing Magazine.

[6]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Toshiaki Fujii,et al.  View Generation with 3D Warping Using Depth Information for FTV , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[8]  Aljoscha Smolic,et al.  Reliability-based generation and view synthesis in layered depth video , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[9]  Harry Shum,et al.  Image-based rendering , 2006, Found. Trends Comput. Graph. Vis..

[10]  Kwanghoon Sohn,et al.  2D/3D freeview video generation for 3DTV system , 2008, 2008 15th IEEE International Conference on Image Processing.

[11]  Aljoscha Smolic,et al.  Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems , 2008, 2008 15th IEEE International Conference on Image Processing.

[12]  Toshiaki Fujii,et al.  Artifact reduction using reliability reasoning for image generation of FTV , 2010, J. Vis. Commun. Image Represent..