Free-Viewpoint TV

Free-viewpoint television (FTV) is an innovative visual media that enables us to view a three-dimensional (3-D) scene by freely changing our viewpoints. We proposed the concept of FTV and constructed the world???s first real-time system including the complete chain of operation from image capture to display. We also carried out the FTV on a single personal computer (PC) and a mobile player. FTV is based on the ray-space method that represents one ray in real space with one point in the ray-space. We have developed several types of ray capture systems and interfaces such as a 360° capture/ray-reproducing display. FTV is regarded as the ultimate 3DTV, since it can generate infinite number of views. Thus, FTV is the key to immersive communication. Regarding FTV as the most challenging 3-D media, the Motion Picture Experts Group (MPEG) has been conducting its international standardization activities. This article reviews FTV and its related technologies.

[1]  Toshiaki Fujii,et al.  The Seelinder: Cylindrical 3D display viewable from 360 degrees , 2010, J. Vis. Commun. Image Represent..

[2]  Toshiaki Fujii,et al.  Multipoint Measuring System for Video and Sound - 100-camera and microphone system , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[3]  Heung-Yeung Shum,et al.  Image-Based Rendering and Synthesis , 2007, IEEE Signal Processing Magazine.

[4]  Toshiaki Fujii,et al.  Experimental system of free viewpoint television , 2003, IS&T/SPIE Electronic Imaging.

[5]  Masayuki Tanimoto,et al.  Frameworks for FTV coding , 2009, 2009 Picture Coding Symposium.

[6]  Toshiaki Fujii,et al.  View generation with 3D warping using depth information for FTV , 2009, Signal Process. Image Commun..

[7]  Toshiaki Fujii,et al.  Free viewpoint TV system based on ray-space representation , 2002, SPIE ITCom.

[8]  Toshiaki Fujii,et al.  Probabilistic reliability based view synthesis for FTV , 2010, 2010 IEEE International Conference on Image Processing.

[9]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[10]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Toshiaki Fujii,et al.  Ray-space acquisition system of all-around convergent views using a rotation mirror , 2007, SPIE Optics East.

[12]  Toshiaki Fujii,et al.  Real-time arbitrary view interpolation and rendering system using ray-space , 2005, SPIE Optics East.

[13]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.

[14]  Toshiaki Fujii,et al.  Novel view synthesis with residual error feedback for FTV , 2010, Electronic Imaging.

[15]  Iso Iec Jtc Sc Wg.  Introduction to Multi-view Video Coding , 2005 .

[16]  Masayuki Tanimoto.  Overview of free viewpoint television , 2006, Signal Process. Image Commun..

[17]  Toshiaki Fujii,et al.  Free viewpoint image generation using multi-pass dynamic programming , 2007, Electronic Imaging.

[18]  Wei-Chao Chen,et al.  Light field mapping: efficient representation and hardware rendering of surface light fields , 2002, SIGGRAPH.

[19]  Toshiaki Fujii,et al.  Artifact reduction using reliability reasoning for image generation of FTV , 2010, J. Vis. Commun. Image Represent..

[20]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[21]  Toshiaki Fujii,et al.  Ray space coding for 3D visual communication , 1996 .

[22]  Masayuki Tanimoto.  Free-Viewpoint Television , 2010, Image and Geometry Processing for 3-D Cinematography.

[23]  Toshiaki Fujii,et al.  A real-time ray-space acquisition system , 2004, IS&T/SPIE Electronic Imaging.

[24]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.