A signal-processing framework for reflection

We present a signal-processing framework for analyzing the reflected light field from a homogeneous convex curved surface under distant illumination. This analysis is of theoretical interest in both graphics and vision and is also of practical importance in many computer graphics problems---for instance, in determining lighting distributions and bidirectional reflectance distribution functions (BRDFs), in rendering with environment maps, and in image-based rendering. It is well known that under our assumptions, the reflection operator behaves qualitatively like a convolution. In this paper, we formalize these notions, showing that the reflected light field can be thought of in a precise quantitative way as obtained by convolving the lighting and BRDF, i.e. by filtering the incident illumination using the BRDF. Mathematically, we are able to express the frequency-space coefficients of the reflected light field as a product of the spherical harmonic coefficients of the illumination and the BRDF. These results are of practical importance in determining the well-posedness and conditioning of problems in inverse rendering---estimation of BRDF and lighting parameters from real photographs. Furthermore, we are able to derive analytic formulae for the spherical harmonic coefficients of many common BRDF and lighting models. From this formal analysis, we are able to determine precise conditions under which estimation of BRDFs and lighting distributions are well posed and well-conditioned. Our mathematical analysis also has implications for forward rendering---especially the efficient rendering of objects under complex lighting conditions specified by environment maps. The results, especially the analytic formulae derived for Lambertian surfaces, are also relevant in computer vision in the areas of recognition, photometric stereo and structure from motion.

[1]  Li Zhang,et al.  Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multiview stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[2]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Shree K. Nayar,et al.  Practical Rendering of Multiple Scattering Effects in Participating Media , 2004, Rendering Techniques.

[4]  J. Arvo Analytic methods for simulated light transport , 1995 .

[5]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[7]  Paul S. Heckbert,et al.  Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter , 1986, IEEE Computer Graphics and Applications.

[8]  Ronen Basri,et al.  Dense shape reconstruction of a moving object under arbitrary, unknown lighting , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Ronen Basri,et al.  Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting , 2004, ECCV.

[10]  HanrahanPat,et al.  A signal-processing framework for reflection , 2004 .

[11]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[12]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[13]  T. Inui,et al.  Group theory and its applications in physics , 1990 .

[14]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[15]  Norman J. McCormick,et al.  Inverse radiative transfer problems : a review , 1992 .

[16]  Katsushi Ikeuchi,et al.  Determining Reflectance Parameters and Illumination Distribution from a Sparse Set of Images for View-dependent Image Synthesis , 2001, ICCV.

[17]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[18]  J. A. Cochran The analysis of linear integral equations , 1973 .

[19]  Katsushi Ikeuchi,et al.  Illumination Distribution from Brightness in Shadows: Adaptive Estimation of Illumination Distribution with Unknown Reflectance Properties in Shadow Regions , 1999, ICCV.

[20]  Hans-Peter Seidel,et al.  Unified Approach to Prefiltered Environment Maps , 2000, Rendering Techniques.

[21]  K. Ikeuchi,et al.  Reflectance analysis under solar illumination , 1994, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[22]  Shree K. Nayar,et al.  Analytic Rendering of Multiple Scattering in Participating Media , 2004 .

[23]  Jan-Olof Eklundh,et al.  Low-dimensional representations of shaded surfaces under varying illumination , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[24]  Robert R. Lewis,et al.  Making Shaders More Physically Plausible , 1994, Comput. Graph. Forum.

[25]  David J. Kriegman,et al.  Nine points of light: acquiring subspaces for face recognition under variable lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[26]  Henrik Wann Jensen,et al.  A rapid hierarchical rendering technique for translucent materials , 2005, SIGGRAPH Courses.

[27]  Stephen H. Westin,et al.  Predicting reflectance functions from complex surfaces , 1992, SIGGRAPH.

[28]  Hans-Peter Seidel,et al.  Spherical harmonic gradients for mid-range illumination , 2004 .

[29]  Stephen H. Westin,et al.  Image-based bidirectional reflectance distribution function measurement. , 2000, Applied optics.

[30]  H. Groemer Geometric Applications of Fourier Series and Spherical Harmonics , 1996 .

[31]  R. Love Surface reflection model estimation from naturally illuminated image sequences , 1997 .

[32]  Nelson L. Max,et al.  Bidirectional reflection functions from surface bump maps , 1987, SIGGRAPH.

[33]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[34]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  L. M. M.-T. Spherical Harmonics: an Elementary Treatise on Harmonic Functions, with Applications , 1928, Nature.

[36]  Lei Zhang,et al.  Face recognition under variable lighting using harmonic image exemplars , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[37]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[38]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Ravi Ramamoorthi,et al.  Analytic PCA Construction for Theoretical Analysis of Lighting Variability in Images of a Lambertian Object , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Stephen H. Westin,et al.  A global illumination solution for general reflectance distributions , 1991, SIGGRAPH.

[41]  James F. Blinn,et al.  Texture and reflection in computer generated images , 1998 .

[42]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  Russell A. Epstein,et al.  5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[44]  J. Koenderink,et al.  Phenomenological description of bidirectional surface reflection , 1998 .

[45]  Jan Kautz,et al.  Approximation of Glossy Reflection with Prefiltered Environment Maps , 2000, Graphics Interface.

[46]  François X. Sillion,et al.  Fast calculation of soft shadow textures using convolution , 1998, SIGGRAPH.

[47]  Ned Greene,et al.  Environment Mapping and Other Applications of World Projections , 1986, IEEE Computer Graphics and Applications.

[48]  Pat Hanrahan,et al.  A signal-processing framework for forward and inverse rendering , 2002 .

[49]  R. Preisendorfer,et al.  Hydrologic optics. Volume 1: Introduction , 1976 .

[50]  Jan-Olof Eklundh,et al.  Phenomenological eigenfunctions for image irradiance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[51]  Jan-Olof Eklundh,et al.  Classifying Materials from Their Reflectance Properties , 2004, ECCV.

[52]  Pat Hanrahan,et al.  All-frequency shadows using non-linear wavelet lighting approximation , 2003, ACM Trans. Graph..

[53]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[54]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[55]  Pat Hanrahan,et al.  Analysis of planar light fields from homogeneous convex curved surfaces under distant illumination , 2001, IS&T/SPIE Electronic Imaging.

[56]  Jitendra Malik,et al.  Recovering photometric properties of architectural scenes from photographs , 1998, SIGGRAPH.

[57]  Katsushi Ikeuchi,et al.  Appearance sampling for obtaining a set of basis images for variable illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[58]  James T. Kajiya,et al.  The rendering equation , 1998 .

[59]  Edward H. Adelson,et al.  Estimating surface reflectance properties from images under unknown illumination , 2001, IS&T/SPIE Electronic Imaging.

[60]  D. A. Dunnett Classical Electrodynamics , 2020, Nature.

[61]  J. Koenderink,et al.  Optical properties (bidirectional reflection distribution functions) of velvet. , 1998, Applied optics.

[62]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[63]  Ravi Ramamoorthi,et al.  A Fourier Theory for Cast Shadows , 2004, ECCV.

[64]  Steve Marschner,et al.  Inverse Lighting for Photography , 1997, CIC.

[65]  Julie Dorsey,et al.  Effic ient Re-rendering of Naturally Illuminated Environments , 1994 .

[66]  Pat Hanrahan,et al.  Frequency space environment map rendering , 2002, SIGGRAPH.

[67]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[68]  Li Zhang,et al.  Shape and motion under varying illumination , 2003, IEEE International Conference on Computer Vision.

[69]  Jan Kautz,et al.  Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments , 2002 .

[70]  Michael F. Cohen,et al.  Radiosity and realistic image synthesis , 1993 .

[71]  Peter W. Hallinan A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[72]  K. Torrance,et al.  Image-Based BRDF Measurement , 1999 .

[73]  C. R. Hoffman,et al.  Illumination and Reflection Maps : Simulated Objects in Simulated and Real Environments Gene , 1984 .

[74]  F. E. Nicodemus,et al.  Geometrical considerations and nomenclature for reflectance , 1977 .

[75]  Hans-Peter Seidel,et al.  Realistic, hardware-accelerated shading and lighting , 1999, SIGGRAPH.

[76]  Kosuke Sato,et al.  Determining Reflectance Properties of an Object Using Range and Brightness Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[77]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[78]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[79]  P. Hanrahan,et al.  Triple product wavelet integrals for all-frequency relighting , 2004, SIGGRAPH 2004.

[80]  Pat Hanrahan,et al.  An efficient representation for irradiance environment maps , 2001, SIGGRAPH.

[81]  Peter-Pike J. Sloan,et al.  Clustered principal components for precomputed radiance transfer , 2003, ACM Trans. Graph..

[82]  Sumanta N. Pattanaik,et al.  Eurographics Symposium on Rendering (2004) a Novel Hemispherical Basis for Accurate and Efficient Rendering , 2022 .

[83]  Marc Olano,et al.  Reflection space image based rendering , 1999, SIGGRAPH.