Image-based modeling, rendering, and lighting

n the past seven years, the adjective image-based has become a common fixture in the computer graphics vernacular. As a result, we've witnessed an explosion of new and original uses for photographs and other forms of acquired images in computer graphics applications. We can use collections of images to relight scenes, capture the subtle appearance qualities of real-world objects, derive geometric models, and synthesize images from original viewpoints. Not coincidentally, this explosion has occurred in conjunction with the availability of affordable digital photography. This special issue of IEEE Computer Graphics and Applications spotlights both the progress and recent advances in image-based approaches to computer graphics. It brings together a range of diverse points of view and demonstrates the indelible impact these approaches have had on our field. The introduction of image-based approaches exemplifies one of many dramatic yet, in hindsight, obvious departures in computer graphics. The quest for photo-realistic renderings has long been the holy grail of computer graphics. Prior to the introduction of image-based methods, the most popular approach to achieving improved realism was to incorporate more and more knowledge of physics into simulation-based rendering systems. Generally, these improvements came with the costs of increased simulation times and the introduction of new, often difficult to obtain, material parameters. Image-based rendering is a refreshing alternative because it suggests that we might distill the essence of realism from example photographs. Furthermore, it's turned the computational focus away from simulation, toward a more manageable interpolation-based approach. Image-based computer graphics techniques started gaining traction as a result of two important leaps. First was the realization that we could incorporate images as first-class components of an underlying scene representation. This contrasted with the prevailing practice where images played a more subordinate role, most frequently as static texture maps draped over geometry. A second realization was the recognition that we could treat images as actual measurements of both geometric and radiometric information. This new interpretation has proven to be rich with new opportunities, including new approaches to visibility, improved methods for modeling view-dependent variations in the appearance of materials, and the possibility of combining radiant energy measurements from different images to reilluminate scenes. Furthermore, this interpretation of images as measurements has fostered new connections between the computer graphics and computer vision communities, as well as provided new perspectives on the problems of inverse rendering. This special issue demonstrates the wide-ranging impact that image-based approaches have …