A layered method of visibility resolving in depth image-based rendering

A layered method is presented in this paper to resolve the visibility problem in depth image-based rendering. A novel three-layer representation for each reference view, i.e. the main layer, background layer and the boundary layer, is proposed. A spatial-temporal method is designed to generate the boundary layer for pixel-based rendering (splatting). Meanwhile, a temporal background model is built for each frame by searching backward and forward for uncovered background information based on depth variance in the reference video. Promising results of view synthesis using the multiple-view 3d data from Microsoft Research, ldquobreak dancerrdquo and ldquoballetrdquo, are given to demonstrate the performance of the proposed method.

[1]  Harry Shum,et al.  Background Cut , 2006, ECCV.

[2]  Tsuhan Chen,et al.  A survey on image-based rendering - representation, sampling and compression , 2004, Signal Process. Image Commun..

[3]  William R. Mark,et al.  Efficient Reconstruction Techniques for Post-Rendering 3D Image Warping , 1998 .

[4]  Ze-Nian Li,et al.  Review and Preview: Disocclusion by Inpainting for Image-Based Rendering , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[6]  Eli Shechtman,et al.  Space-time video completion , 2004, CVPR 2004.

[7]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[9]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[10]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[11]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[12]  Andrew Blake,et al.  Efficient Dense Stereo with Occlusions for New View-Synthesis by Four-State Dynamic Programming , 2006, International Journal of Computer Vision.

[13]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[14]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[15]  Marc Levoy,et al.  The Use of Points as a Display Primitive , 2000 .

[16]  Leonard McMillan,et al.  Head-tracked stereoscopic display using image warping , 1995, Electronic Imaging.

[17]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.