The plenoptic videos: capturing, rendering and compression

This paper presents a system for capturing and rendering a dynamic image-based representation called the plenoptic videos. It is a simplified version of light fields for dynamic environment, where user viewpoints are constrained along the camera plane of a linear array of video cameras. The system consists of a camera array of 8 Sony CCX-Z11 CCD cameras and eight Pentium 41.8 GHz computers connected together through a 100 baseT LAN. Important issues such as multiple camera calibration, real-time compression, decompression and rendering are addressed. Experimental results demonstrated the usefulness of the proposed parallel processing based system in capturing and rendering high quality dynamic image-based representation using off-the-shelf equipment, and its potential applications in visualization and immersive television systems.

[1]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[2]  Mark A. Horowitz,et al.  Light field video camera , 2000, IS&T/SPIE Electronic Imaging.

[3]  Takeshi Naemura,et al.  Real-Time Video-Based Modeling and Rendering of 3D Scenes , 2002, IEEE Computer Graphics and Applications.

[4]  Leonard McMillan,et al.  A Real-Time Distributed Light Field Camera , 2002, Rendering Techniques.

[5]  Antonio Ortega,et al.  Stereo image compression with disparity compensation using the MRF model , 1996, Other Conferences.

[6]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[7]  Harry Shum,et al.  Virtual reality using the concentric mosaic: construction, rendering and data compression , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[8]  Harry Shum,et al.  On the data compression and transmission aspects of panoramic video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[9]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[10]  Harry Shum,et al.  The compression of simplified dynamic light fields , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[11]  Jin Li,et al.  Compression of Lumigraph with multiple reference frame (MRF) prediction and just-in-time rendering , 2000, Proceedings DCC 2000. Data Compression Conference.

[12]  Harry Shum,et al.  On the Compression of Image Based Rendering Scene: A Comparison among Block, Reference and Wavelet Coders , 2001, Int. J. Image Graph..

[13]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[14]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[15]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Xin Tong,et al.  Coding of multi-view images for immersive viewing , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[17]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[18]  Marcus A. Magnor,et al.  Data compression for light-field rendering , 2000, IEEE Trans. Circuits Syst. Video Technol..

[19]  Marcus A. Magnor,et al.  Hardware-Accelerated Dynamic Light Field Rendering , 2002, VMV.