A System for Capturing, Rendering and Multiplexing Images on Multi-view Autostereoscopic Display

Current trends in digital display technology show a marked interest towards 3D displays, which allow three dimensional images to be conveyed to viewers. Among various 3D display techniques, auto stereoscopic display appears to be promising due to the use of optical trickery at the display, allowing glass-free viewing. However, the cost of generating and transmitting the auto stereoscopic images is usually quite high due to the huge amount of data. Hence, it is challenging to acquire artifact-free 3D images in real-time. This paper presents a system to generate and display high-resolution auto stereoscopic images at full HD resolution, i.e., 1920*1080*24. We show that the video-plus-depth data representation enables a scalable system architecture and efficient data transition. The proposed GPU accelerated depth image-based rendering (DIBR) algorithm and multiplexing algorithm are able to synthesize auto stereoscopic images in real-time. The synthesized images are then displayed on the auto stereoscopic screen that is mounted on a conventional LCD monitor. We demonstrate our system to both indoor activities and natural scenes.

[1]  Martin G. H. Hiddink,et al.  2D/3D switchable displays , 2006, SPIE Photonics Europe.

[2]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[3]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Manuel Menezes de Oliveira Neto,et al.  Relief mapping of non-height-field surface details , 2006, I3D '06.

[5]  Neil A. Dodgson,et al.  Autostereoscopic 3D displays , 2005, Computer.

[6]  Aljoscha Smolic,et al.  3DAV exploration of video-based rendering technology in MPEG , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[8]  Szymon Rusinkiewicz,et al.  Spacetime Stereo: A Unifying Framework for Depth from Triangulation , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[10]  Jonathan Mather,et al.  2D/3D Switchable Displays (特集1 液晶要素技術) , 2003 .

[11]  Shing-Tung Yau,et al.  High-resolution, real-time-geometry video acquisition system , 2006, SIGGRAPH '06.

[12]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[13]  Song Zhang,et al.  High-resolution, real-time 3D absolute coordinate measurement based on a phase-shifting method. , 2006, Optics express.

[14]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[15]  Joshua Napoli,et al.  Volumetric three-dimensional display system with rasterization hardware , 2001, IS&T/SPIE Electronic Imaging.

[16]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[17]  Armin Grasnick,et al.  Multiviewpoint autostereoscopic dispays from 4D-Vision GmbH , 2002, IS&T/SPIE Electronic Imaging.

[18]  Wojciech Matusik,et al.  3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes , 2004, ACM Trans. Graph..

[19]  Song Zhang,et al.  High-Resolution, Real-time 3D Shape Acquisition , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[20]  Song Zhang,et al.  High-resolution, real-time three-dimensional shape measurement , 2006 .

[21]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[22]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[23]  Steven M. Seitz,et al.  Spacetime faces , 2004, ACM Trans. Graph..

[24]  S Nakajima,et al.  Three-dimensional medical imaging display with computer-generated integral photography. , 2001, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[25]  L. Lipton.  Foundations of the Stereoscopic Cinema , 1982 .

[26]  Jens-Rainer Ohm,et al.  A realtime hardware system for stereoscopic videoconferencing with viewpoint adaptation , 1998, Signal Process. Image Commun..

[27]  Song Zhang.  Recent progresses on real-time 3D shape measurement using digital fringe projection techniques , 2010 .

[28]  Stephen Lin,et al.  Generalized Displacement Maps , 2004, Rendering Techniques.

[29]  Koki Sato,et al.  Electro-holographic display using 15mega pixels LCD , 1996, Electronic Imaging.

[30]  Liang-Gee Chen,et al.  Real-Time Depth Image based Rendering Hardware Accelerator for Advanced Three Dimensional Television System , 2006, 2006 IEEE International Conference on Multimedia and Expo.