Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV

This paper presents details of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework. The work is part of the European Information Society Technologies (IST) project "Advanced Three-Dimensional Television System Technologies" (ATTEST), an activity, where industries, research centers and universities have joined forces to design a backwards-compatible, flexible and modular broadcast 3D-TV system. At the very heart of the described new concept is the generation and distribution of a novel data representation format, which consists of monoscopic color video and associated per-pixel depth information. From these data, one or more "virtual" views of a real-world scene can be synthesized in real-time at the receiver side (i.e. a 3D-TV set-top box) by means of so-called depth-image-based rendering (DIBR) techniques. This publication will provide: (1) a detailed description of the fundamentals of this new approach on 3D-TV; (2) a comparison with the classical approach of "stereoscopic" video; (3) a short introduction to DIBR techniques in general; (4) the development of a specific DIBR algorithm that can be used for the efficient generation of high-quality "virtual" stereoscopic views; (5) a number of implementation details that are specific to the current state of the development; (6) research on the backwards-compatible compression and transmission of 3D imagery using state-of-the-art MPEG (Moving Pictures Expert Group) tools.

[1]  Wijnand A. IJsselsteijn,et al.  Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain , 2000, Electronic Imaging.

[2]  Marc Pollefeys,et al.  An evolutionary and optimised approach on 3D-TV , 2002 .

[3]  G. Iddan,et al.  3D IMAGING IN THE STUDIO (AND ELSEWHERE...) , 2001 .

[4]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[5]  Peter Kauff,et al.  Interactive Virtual View Video (IVVV) - The Bridge Between Immersive TV and 3D-TV , 2002, SPIE ITCom.

[6]  Gang Xu,et al.  Epipolar Geometry in Stereo, Motion and Object Recognition , 1996, Computational Imaging and Vision.

[7]  Oliver Schreer,et al.  3D analysis and image-based rendering for immersive TV applications , 2002, Signal Process. Image Commun..

[8]  George Wolberg,et al.  Digital image warping , 1990 .

[9]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[10]  C. Fehn.  A 3 DTV Approach Using Depth-Image-Based Rendering ( DIBR ) , 2003 .

[11]  Eyal Ofek,et al.  Depth keying , 2003, IS&T/SPIE Electronic Imaging.

[12]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[13]  Thomas E. Dawson,et al.  The Effects of Age upon the Perception of Depth and 3-D Shape from Differential Motion and Binocular Disparity , 2000, Perception.

[14]  Lenny Lipton,et al.  Foundations of the stereoscopic cinema : a study in depth , 1984 .

[15]  William R. Mark,et al.  Post-Rendering 3D Image Warping: Visibility, Reconstruction, and Performance for Depth-Image Warping , 1999 .

[16]  Andrew J. Woods,et al.  Image distortions in stereoscopic video systems , 1993, Electronic Imaging.

[17]  Iain E. G. Richardson,et al.  Video CODEC Design , 2002 .

[18]  Kostas Daniilidis,et al.  View-independent scene acquisition for tele-presence , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[19]  Marc Pollefeys.  3D Modelling from Images , 2000, ECCV 2000.

[20]  Paul Milgram,et al.  Adaptation effects in stereo due to on-line changes in camera configuration , 1992, Electronic Imaging.

[21]  D. Runde,et al.  How to realize a natural image reproduction using stereoscopic displays with motion parallax , 2000, IEEE Trans. Circuits Syst. Video Technol..