DATA FORMAT AND CODING FOR FREE VIEWPOINT VIDEO

In this paper we will discuss data representation formats and coding aspects of a new 3D functionality called Free Viewpoint Video (FVV). Similar to Computer Graphics (CG) or VRML applications, FVV allows for navigating with a virtual camera in 3D scenes. However, in contrast to graphical scenes in CG and VRML, FVV is based on natural scenes consisting of 3D video objects that provide a higher level of realism and allow a 3D reproduction of real events. Against this background the paper will review different approaches on FVV and different 3D representation formats. It will discuss the acquisition of required representation parameters and related rendering aspects. Furthermore it will present results of coding experiments for two particular 3D representation formats.

[1]  Xiaojun Wu,et al.  Real-time dynamic 3-D object shape reconstruction and high-fidelity texture mapping for 3-D video , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Klaus Hopf,et al.  Key technologies for an advanced 3D TV system , 2004, SPIE Optics East.

[3]  Amnon Shashua,et al.  Novel View Synthesis by Cascading Trilinear Tensors , 1998, IEEE Trans. Vis. Comput. Graph..

[4]  Volkan Isler,et al.  Trinocular Stereo: A Real-Time Algorithm and its Evaluation , 2004, International Journal of Computer Vision.

[5]  Aljoscha Smolic,et al.  3DAV exploration of video-based rendering technology in MPEG , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Wei-Chao Chen,et al.  Light field mapping: efficient representation and hardware rendering of surface light fields , 2002, SIGGRAPH.

[7]  Aljoscha Smolic,et al.  Multi-texture modeling of 3D traffic scenes , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[8]  Manuel Menezes de Oliveira Neto,et al.  Relief texture mapping , 2000, SIGGRAPH.

[9]  Aljoscha Smolic,et al.  Interactive 3-D Video Representation and Coding Technologies , 2005, Proceedings of the IEEE.

[10]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[11]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[12]  Richard Szeliski,et al.  The geometry-image representation tradeoff for rendering , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[13]  Markus H. Gross,et al.  3D video fragments: dynamic point samples for real-time free-viewpoint video , 2004, Comput. Graph..

[14]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[15]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[16]  K. Mueller.  CODING OF 3D MESHES AND VIDEO TEXTURES FOR 3D VIDEO OBJECTS , 2004 .

[18]  Marcus A. Magnor,et al.  Real-time microfacet billboarding for free-viewpoint video rendering , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[19]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.