Am object-based approach to plenoptic videos

This paper proposes an object-based approach to plenoptic videos, where the plenoptic video sequences are segmented into image-based rendering (IBR) objects each with its image sequence, depth map and other relevant information such as shape information. This allows desirable functionalities such as scalability of contents, error resilience, and interactivity with individual IBR objects to be supported. A portable capturing system consisting of two linear camera arrays, each hosting 6 JVC video cameras, was developed to verify the proposed approach. Rendering and compression results of real-world scenes demonstrate the usefulness and good quality of the proposed approach.

[1]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[2]  Leonard McMillan,et al.  A Real-Time Distributed Light Field Camera , 2002, Rendering Techniques.

[3]  Harry Shum,et al.  On the data compression and transmission aspects of panoramic video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[4]  Harry Shum,et al.  The plenoptic videos: capturing, rendering and compression , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[5]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[6]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[7]  Mark A. Horowitz,et al.  Light field video camera , 2000, IS&T/SPIE Electronic Imaging.

[8]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[9]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[10]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[11]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[13]  Harry Shum,et al.  Pop-up light field: An interactive image-based modeling and rendering system , 2004, TOGS.

[14]  Harry Shum,et al.  The compression of simplified dynamic light fields , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[15]  Marcus A. Magnor,et al.  Hardware-Accelerated Dynamic Light Field Rendering , 2002, VMV.

[16]  Harry Shum,et al.  On the rendering and post-processing of simplified dynamic light fields with depth information , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[17]  Takeshi Naemura,et al.  Real-Time Video-Based Modeling and Rendering of 3D Scenes , 2002, IEEE Computer Graphics and Applications.

[18]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.