Authoring effective depictions of reality by combining multiple samples of the plenoptic function

Cameras are powerful tools for depicting the world around us, but the images they produce are interpretations of reality that often fail to resemble our intended interpretation. While photographs provide a strong illusion of realism, they are not always effective at depicting what the author hoped to depict. The recent digitization of photography and video offers the opportunity to move beyond the traditional constraints of analog photography and improve the techniques we use to create depictions from light in a scene. In this thesis, I explore one approach to addressing this challenge. My approach begins by capturing multiple digital photographs and videos of a scene. I present algorithms and interfaces that identify the best pieces of this captured imagery, as well as algorithms for seamlessly fusing these pieces into new depictions that are better than any of the originals. As I show in this thesis, the results are often much more effective than what could be achieved using traditional photographic techniques. I apply this approach to three projects in particular. The "photomontage" system is an interactive tool that partially automates the process of combining the best features of a stack of images. The user authors a composite image by specifying high-level objectives that the composite should exhibit; the system then constructs a depiction from pieces of the input images by maximizing these objectives while also minimizing visual artifacts. In the next project, I extend the photomontage approach to handle sequences of photographs captured from shifted viewpoints. The output is a "multi-viewpoint panorama" that is useful for depicting scenes too long to effectively image with the single-viewpoint perspective produced by a traditional camera. In the final project, I extend my approach to video sequences. I introduce the "panoramic video texture," which is a video with a wide field of view that appears to play continuously and indefinitely. The result is a new medium that combines the benefits of panoramic photography and video to provide a more immersive depiction of a scene. The key challenge in this project is that panoramic video textures are created from the input of a single video camera that slowly pans across the scene; thus, although only a portion of the scene has been imaged at any given time, the output must simultaneously portray motion throughout the scene. Throughout this thesis, I demonstrate how these novel techniques can be used to create expressive visual media that more effectively depicts reality.