High performance imaging using large camera arrays

The advent of inexpensive digital image sensors and the ability to create photographs that combine information from a number of sensed images are changing the way we think about photography. In this paper, we describe a unique array of 100 custom video cameras that we have built, and we summarize our experiences using this array in a range of imaging applications. Our goal was to explore the capabilities of a system that would be inexpensive to produce in the future. With this in mind, we used simple cameras, lenses, and mountings, and we assumed that processing large numbers of images would eventually be easy and cheap. The applications we have explored include approximating a conventional single center of projection video camera with high performance along one or more axes, such as resolution, dynamic range, frame rate, and/or large aperture, and using multiple cameras to approximate a video camera with a large synthetic aperture. This permits us to capture a video light field, to which we can apply spatiotemporal view interpolation algorithms in order to digitally simulate time dilation and camera motion. It also permits us to create video sequences using custom non-uniform synthetic apertures.

[1]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[2]  Richard Szeliski,et al.  Image mosaicing for tele-reality applications , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[3]  Marc Levoy,et al.  High-speed videography using a dense camera array , 2004, CVPR 2004.

[4]  Leonard McMillan,et al.  A Real-Time Distributed Light Field Camera , 2002, Rendering Techniques.

[5]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, CVPR 2004.

[6]  Tsuhan Chen,et al.  A self-reconfigurable camera array , 2004, SIGGRAPH '04.

[7]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[8]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[9]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[10]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Ye Zhang,et al.  On 3D scene flow and structure estimation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Hai Tao,et al.  A global matching framework for stereo computation , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Yaron Caspi,et al.  Increasing Space-Time Resolution in Video , 2002, ECCV.

[14]  Tsuhan Chen,et al.  View-dependent nonuniform sampling for image-based rendering , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[15]  Mark A. Horowitz,et al.  Light field video camera , 2000, IS&T/SPIE Electronic Imaging.

[16]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[17]  Takeo Kanade,et al.  Virtual ized reality: constructing time-varying virtual worlds from real world events , 1997 .

[18]  Marc Levoy,et al.  Synthetic Aperture Focusing using a Shear-Warp Factorization of the Viewing Transform , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[19]  Richard Szeliski,et al.  High dynamic range video , 2003, ACM Trans. Graph..

[20]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[21]  Mike Tooley,et al.  4 – System architecture and construction , 1999 .

[22]  Leonard McMillan,et al.  A new reconstruction filter for undersampled light fields , 2003 .

[23]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.