3-D scene reconstruction with viewpoint adaptation on stereo displays

We propose a generic algorithm for the geometrically correct reconstruction of 3-D scenes on stereo displays with viewpoint adaptation. This forms the basis of multiviewpoint systems, which are currently the most promising candidates for real-time implementations of 3-D visual communication systems. The reconstruction algorithm needs 3-D tracking of the viewers' eyes with respect to the display. We analyze the effect of eye-tracking errors. A simple bound is derived, below which reconstruction errors cannot be observed. We design a multiviewpoint system using a recently introduced image-based scene representation. The design formed the basis of the real-time multiviewpoint system that was recently built in the European PANORAMA project. Experiments with both natural and synthetic scenes show that the proposed reconstruction algorithm performs well. The experiments are performed by computer simulations and the real-time PANORAMA system.

[1]  Siegmund Pastoor,et al.  3D-television: A survey of recent research results on subjective requirements , 1991, Signal Process. Image Commun..

[2]  Tim J. Dennis,et al.  Epipolar line estimation and rectification for stereo image pairs , 1996, IEEE Trans. Image Process..

[3]  Sing Bing Kang,et al.  Survey of image-based rendering techniques , 1998, Electronic Imaging.

[4]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[5]  Toshiaki Fujii,et al.  Ray space coding for 3D visual communication , 1996 .

[6]  Laurent Vaissie,et al.  Analysis of eyepoint locations and accuracy of rendered depth in binocular head-mounted displays , 1999, Electronic Imaging.

[7]  I. Sexton,et al.  Stereoscopic and autostereoscopic display systems , 1999, IEEE Signal Process. Mag..

[8]  Jens-Rainer Ohm,et al.  A realtime hardware system for stereoscopic videoconferencing with viewpoint adaptation , 1998, Signal Process. Image Commun..

[9]  Stephen Herman,et al.  Principles of Binocular 3D Displays with Applications to Television , 1971 .

[10]  Jeffrey S. McVeigh,et al.  Intermediate view synthesis considering occluded and ambiguously referenced image regions , 1996, Signal Process. Image Commun..

[11]  J. Biemond,et al.  Correspondence estimation in image pairs , 1999, IEEE Signal Process. Mag..

[12]  Aladdin M. Ariyaeeinia.  Analysis and design of stereoscopic television systems , 1998, Signal Process. Image Commun..

[13]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[14]  Jens-Rainer Ohm,et al.  An object-based system for stereoscopic viewpoint synthesis , 1997, IEEE Trans. Circuits Syst. Video Technol..

[15]  B. Wandell.  Foundations of vision , 1995 .

[16]  Emile A. Hendriks,et al.  Synthesis of multi viewpoint images at non-intermediate positions , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[17]  Robert Kutka.  Reconstruction of correct 3-D perception on screens viewed at different distances , 1994, IEEE Trans. Commun..

[18]  Emile A. Hendriks,et al.  Real-time synthesis of digital multiple-viewpoint stereoscopic images , 1999, Electronic Imaging.

[19]  Gregg Podnar,et al.  Geometry of binocular imaging , 1994, Electronic Imaging.

[20]  Janusz Konrad.  Enhancement of viewer comfort in stereoscopic viewing: parallax adjustment , 1999, Electronic Imaging.

[21]  Steven M. Seitz,et al.  Physically-valid view synthesis by image interpolation , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).