Scalable virtual viewpoint image synthesis for multiple camera environments

One of the main aims of emerging audio-visual (AV) applications is to provide interactive navigation within a captured event or scene. This paper presents a view synthesis algorithm that provides a scalable and flexible approach to virtual viewpoint synthesis in multiple camera environments. The multi-view synthesis (MVS) process consists of four different phases that are described in detail: surface identification, surface selection, surface boundary blending and surface reconstruction. MVS view synthesis identifies and selects only the best quality surface areas from the set of available reference images, thereby reducing perceptual errors in virtual view reconstruction. The approach is camera setup independent and scalable as virtual views can be created given 1 to N of the available video inputs. Thus, MVS provides interactive AV applications with a means to handle scenarios where camera inputs increase or decrease over time.

[1]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[2]  Oliver Schreer,et al.  3D analysis and image-based rendering for immersive TV applications , 2002, Signal Process. Image Commun..

[3]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[4]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Aljoscha Smolic,et al.  3DAV exploration of video-based rendering technology in MPEG , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Aljoscha Smolic,et al.  Interactive 3-D Video Representation and Coding Technologies , 2005, Proceedings of the IEEE.

[7]  Oliver Schreer,et al.  IMAGE ANALYSIS AND SYNTHESIS FOR MULTIPLE STEREO CAMERA TELECONFERENCING SYSTEMS , 2003 .

[8]  Aljoscha Smolic,et al.  Multi-texture modeling of 3D traffic scenes , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[9]  Harry Shum,et al.  Review of image-based rendering techniques , 2000, Visual Communications and Image Processing.

[10]  Masayuki Tanimoto.  Free Viewpoint Television (FTV) , 2007 .