3D View Synthesis with Feature-Based Warping

Three-dimensional video (3DV), as the new generation of video format standard, can provide the viewers with a vivid screen sense and a realistic stereo impression. Meanwhile the view synthesis has become an important issue for 3DV application. Differently from the conventional methods based on depth, in this paper we propose a new view synthesis algorithm, which can employ the correlation among views and warp in the image domain only. There are mainly two contributions. One is the incorporation of sobel edge points into feature extraction and matching, which can obtain a better stable homography and then a visual comfortable synthesis view compared to SIFT points only. The other is a novel image blending method proposed to obtain a better synthesis image. Experimental results demonstrate that the proposed method can improve the synthesis quality both in subjectivity and objectivity. robust at the expense of slight delay in time consume. As the second contribution, by using the novel blending, we work out the empty region occurred in the synthesis image by the one-sided view. Experimental results show that the synthesis views generated from our proposed method presents high quality with less ghost artifact.

[1]  Aljoscha Smolic,et al.  Automatic View Synthesis by Image-Domain-Warping , 2013, IEEE Transactions on Image Processing.

[2]  R. Sarpong,et al.  Bio-inspired synthesis of xishacorenes A, B, and C, and a new congener from fuscol† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c9sc02572c , 2019, Chemical science.

[3]  Nikolce Stefanoski,et al.  Automatic content creation for multiview autostereoscopic displays using image domain warping , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Zhanyi Hu,et al.  Robust line matching through line-point invariants , 2012, Pattern Recognit..

[6]  Hao Wang,et al.  Spatio-temporal coherence for 3-D view synthesis with curve-based disparity warping , 2014, 2014 IEEE Visual Communications and Image Processing Conference.

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  Yao Zhao,et al.  Asymmetric multiview image coding based on feature matching , 2014, Photonics Asia.

[9]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[10]  Yang Ping,et al.  A New Inter-View Prediction Method for Multi-View Video Coding , 2007, 2007 IEEE Workshop on Signal Processing Systems.

[11]  Carl J. Debono,et al.  Depth-based image processing for 3d video rendering applications , 2014, IWSSIP 2014 Proceedings.

[12]  Haojie Li,et al.  Novel Coplanar Line-Points Invariants for Robust Line Matching Across Views , 2016, ECCV.

[13]  Markus H. Gross,et al.  Non-linear warping and warp coding for content-adaptive prediction in advanced video coding applications , 2010, 2010 IEEE International Conference on Image Processing.

[14]  K. Mardia,et al.  A review of image-warping methods , 1998 .