A signal-processing framework for forward and inverse rendering

The study of the computational aspects of reflection, and especially the interaction between the illumination and the reflective properties of a surface, is of fundamental importance in both computer graphics and vision. In computer graphics, this interaction is a basic building block in most rendering algorithms, i.e. methods that create synthetic computer-generated images. To obtain accurate illumination and reflectance models for rendering, and in computer vision, we often want to invert this interaction by inverse rendering. This dissertation describes a new way of looking at reflection on a curved surface, as a special type of convolution of the incident illumination and the reflective properties of the surface (technically, the bi-directional reflectance distribution function or BRDF. We first theoretically formalize these ideas, leading to the derivation of a convolution theorem in terms of the spherical harmonic coefficients of the lighting and BRDF. This allows us to introduce a signal-processing framework for reflection, wherein the incident lighting is the signal, the BRDF is the filter, and the reflected light is obtained by filtering the input illumination (signal) using the frequency response of the BRDF filter. We then show how our framework can be used practically for computing and displaying synthetic images in real-time with natural illumination and physically-based BRDFs. Next, we extend and apply our framework to inverse rendering. We demonstrate estimation of realistic lighting and reflective properties from photographs, and show how this approach can be used to synthesize realistic images under novel lighting and viewing conditions.