Computational Cameras: Approaches, Benefits and Limits

A computational camera uses a combination of optics and software to produce images that cannot be taken with traditional cameras. In the last decade, computational imaging has emerged as a vibrant field of research. A wide variety of computational cameras have been demonstrated-some designed to achieve new imaging functionalities, and others to reduce the complexity of traditional imaging. In this article, we describe how computational cameras have evolved and present a taxonomy for the technical approaches they use. We explore the benefits and limits of computational imaging, and discuss how it is related to the adjacent and overlapping fields of digital imaging, computational photography and computational image sensors. Over the last century, the evolution of the camera has been truly remarkable. However, through this evolution the basic model underlying the camera has remained essentially the same, namely, the camera obscura (Figure 1(a)). The traditional camera has a detector and a standard lens which captures only those principal rays that pass through its center of projection, or effective pinhole, to produce the familiar linear perspective image. In other words, the traditional camera performs a very simple and restrictive sampling of the complete set of rays, or the light field, that resides in any real scene. A computational camera (Figure 1(b)) uses a combination of novel optics and computations to produce the final image [Nayar 2006a]. The novel optics is used to map rays in the light field of the scene to pixels on the detector in some unconventional fashion. For instance, the ray shown in Figure 1(b) has been geometrically redirected by the optics to a different pixel from the one it would have arrived at in the case of a traditional camera. As illustrated by the change in color from yellow to red, the ray could also be photometrically altered by the optics. In all cases, the captured image is optically coded and may not be meaningful in its raw form. The computational module has a model of the optics, which it uses to decode the captured image to produce a new type of image that could benefit a vision system. The vision system could either be a human observing the image or a computer vision system that uses the image to interpret the scene it represents. 3 Figure 1: (a) The traditional camera model, which is based on the camera obscura. (b) A computational camera uses optical coding followed …