Mask-based light field capture and display

This dissertation describes light-efficient methods for capturing and displaying 3D images using thin, optically-attenuating masks. Light transport is modeled, under geometrical optics, as a 4D function: the light field; this function records the amount of light traveling through any point along any direction. Conventional photographs only record a 2D projection of the incident light field. Each image point is produced by integrating over the full hemisphere of incidence angles. Similarly, conventional displays only approximate a diffuse surface, where the amount of light leaving any point is constant over the full hemisphere of viewing angles. Thus, conventional cameras and displays only support 2D images, for which the perception of scene depth is lost. 3D images can be captured and displayed by including masks in conventional camera and display architectures. Parallax barriers are one example; a mask containing a uniform array of slits is placed slightly in front of a conventional display. This mask only allows certain disjoint display regions to be visible from each viewpoint. 3D image capture is achieved by placing a similar mask close to a sensor. In both cases, 3D images come at the cost of decreased resolution and brightness. This dissertation presents a first-principles analysis of dual-layer camera and display architectures, wherein the first layer is a conventional sensor or display and the second layer is a mask. Novel masks are developed that facilitate 3D image capture and display, outperforming conventional parallax barriers in terms of total light transmission and light field resolution. For 3D capture, a family of static, periodic, non-adaptive masks are derived from a frequency-domain analysis. For 3D display, a linear algebraic analysis reveals a set of time-multiplexed, aperiodic, adaptive masks. Four motivating applications are presented: digital photography, single-shot visual hull reconstruction, depth-sensing LCDs, and 3D display using dual-stacked LCDs.