View and Depth Preprocessing for View Synthesis Enhancement

In the paper, two preprocessing methods for virtual view synthesis are presented. In the first approach, both horizontal and vertical resolutions of the real views and the corresponding depth maps are doubled in order to perform view synthesis on images with densely arranged points. In the second method, real views are filtered in order to eliminate blurred or improperly shifted edges of the objects. Both methods are performed prior to synthesis, thus they may be applied to different Depth-Image-Based Rendering algorithms. In the paper, for both proposed methods, the achieved quality gains are presented.

[1]  Gauthier Lafruit,et al.  Robust Multiview Synthesis for Wide-Baseline Camera Arrays , 2018, IEEE Transactions on Multimedia.

[2]  Marek Domanski,et al.  Enhancing view synthesis with image and depth map upsampling , 2017, 2017 International Conference on Systems, Signals and Image Processing (IWSSIP).

[3]  Peter H. N. de With,et al.  Free-viewpoint depth image based rendering , 2010, J. Vis. Commun. Image Represent..

[4]  Changick Kim,et al.  A Novel Depth-Based Virtual View Synthesis Method for Free Viewpoint Video , 2013, IEEE Transactions on Broadcasting.

[5]  Marc Pollefeys,et al.  Multiple view geometry , 2005 .

[6]  Philippe Bekaert,et al.  An end-to-end system for free viewpoint video for smooth camera transitions , 2012, 2012 International Conference on 3D Imaging (IC3D).

[7]  Yo-Sung Ho,et al.  Boundary Filtering on Synthesized Views of 3D Video , 2008, 2008 Second International Conference on Future Generation Communication and Networking Symposia.

[8]  Masayuki Tanimoto Overview of FTV (free-viewpoint television) , 2010, Visual Communications and Image Processing.

[9]  Toshiaki Fujii,et al.  Multipoint Measuring System for Video and Sound - 100-camera and microphone system , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[10]  Cheolkon Jung,et al.  Real-Time Depth-Image-Based Rendering on GPU , 2015, 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery.

[11]  Marek Domanski,et al.  View Synthesis for Multiview Video Transmission , 2009, IPCV.

[12]  Aljoscha Smolic,et al.  View Synthesis for Advanced 3D Video Systems , 2008, EURASIP J. Image Video Process..

[13]  Krzysztof Wegner,et al.  Experiments on acquisition and processing of video for free-viewpoint television , 2014, 2014 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[14]  Patrik Goorts Real-time, Adaptive Plane Sweeping for Free Viewpoint Navigation in Soccer Scenes , 2014 .

[15]  Olgierd Stankiewicz,et al.  A Free-Viewpoint Television System for Horizontal Virtual Navigation , 2018, IEEE Transactions on Multimedia.

[16]  Takanori Senoh,et al.  New visual coding exploration in MPEG: Super-MultiView and Free Navigation in Free viewpoint TV , 2016, SD&A.

[17]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[18]  Krzysztof Wegner,et al.  Multiview synthesis — Improved view synthesis for virtual navigation , 2016, 2016 Picture Coding Symposium (PCS).

[19]  View Synthesis Reference Software (VSRS) 4.2 with improved inpainting and hole filing , 2017 .

[20]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[21]  Toshiaki Fujii,et al.  FTV for 3-D Spatial Communication , 2012, Proceedings of the IEEE.