A Real-Time Distributed Light Field Camera

We present the design and implementation of a real-time, distributed light field camera. Our system allows multiple viewers to navigate virtual cameras in a dynamically changing light field that is captured in real-time. Our light field camera consists of 64 commodity video cameras that are connected to off-the-shelf computers. We employ a distributed rendering algorithm that allows us to overcome the data bandwidth problems inherent in dynamic light fields. Our algorithm works by selectively transmitting only those portions of the video streams that contribute to the desired virtual views. This technique not only reduces the total bandwidth, but it also allows us to scale the number of cameras in our system without increasing network bandwidth. We demonstrate our system with a number of examples.

[1]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[2]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[3]  Takeshi Naemura,et al.  Pixel independent random access image sensor for real time image-based rendering system , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[4]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[5]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[6]  Hans-Peter Seidel,et al.  On‐the‐Fly Processing of Generalized Lumigraphs , 2001, Comput. Graph. Forum.

[7]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[8]  Takeshi Naemura,et al.  Real-Time Video-Based Modeling and Rendering of 3D Scenes , 2002, IEEE Computer Graphics and Applications.

[9]  S. B. Kang,et al.  Recovering 3 D Shape and Motion from Image Streams using Non-Linear Least Squares , 1993 .

[10]  Gavin S. P. Miller,et al.  A real-time low-latency hardware light-field renderer , 1999, SIGGRAPH.

[11]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[12]  Mark A. Horowitz,et al.  Light field video camera , 2000, IS&T/SPIE Electronic Imaging.

[13]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Takeo Kanade,et al.  Appearance-based virtual view generation of temporally-varying events from multi-camera images in the 3D room , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[15]  Peter-Pike J. Sloan,et al.  Time critical lumigraph rendering , 1997, SI3D.

[16]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[17]  Takeshi Naemura,et al.  Real-time video-based rendering for augmented spatial communication , 1998, Electronic Imaging.

[18]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[19]  Ross Cutler,et al.  Practical calibrations for a real-time digital omnidirectional camera , 2001 .