Efficiently acquiring reflectance fields using patterned illumination

The use of reflectance fields of real world objects to render realistic looking images is increasing. The reflectance field describes the transport of light between the light incident on an object and the light exitant from it. This has numerous applications in areas like entertainment, cultural heritage, digital libraries and space exploration. The central problem with this approach is the lack of fast methods to acquire reflectance fields. This dissertation describes a system for acquiring reflectance field of real world objects that performs many orders of magnitude faster than previous approaches. The system models the 8D reflectance field as a transport matrix between the 4D incident light field and the 4D exitant light field. It is a challenge to measure this matrix because of its large size. However, occasionally the matrix is sparse, e.g. in scenes with little or no inter-reflections. To measure such matrices, this thesis describes a hierarchical technique called dual photography which exploits this sparseness to parallelize the acquisition. This technique, however, performs poorly for scenes with significant diffuse inter-reflections because in such cases the matrix is dense. Fortunately, in these cases the matrix is often data-sparse. Data-sparseness means that sub-blocks of the matrix can be well approximated using low-rank representations. Additionally, the transport matrix is symmetric. Symmetry enables simultaneous measurements from both sides, rows and columns, of the transport matrix. These measurements are used to develop a hierarchical algorithm that can exploit the data-sparseness by a local rank-1 approximation. This technique, called symmetric photography, parallelizes acquisition for dense but data-sparse transport matrices. In the process, this thesis introduces hierarchical tensors as the underlying data structure to represent data-sparse matrices. Besides providing an efficient representation for storage, it enables fast acquisition of the transport matrix and fast rendering of images from the captured matrix. The prototype acquisition system consists of an array of mirrors and a pair of coaxial projector and camera controlled by a computer. The system's effectiveness is demonstrated with scenes rendered from reflectance fields that were captured by it. In these renderings one can change the viewpoint as well as relight objects arbitrarily.