What an image reveals about material reflectance

We derive precise conditions under which material reflectance properties may be estimated from a single image of a homogeneous curved surface (canonically a sphere), lit by a directional source. Based on the observation that light is reflected along certain (a priori unknown) preferred directions such as the half-angle, we propose a semiparametric BRDF abstraction that lies between purely parametric and purely data-driven models. Formulating BRDF estimation as a particular type of semiparametric regression, both the preferred directions and the form of BRDF variation along them can be estimated from data. Our approach has significant theoretical, algorithmic and empirical benefits, lends insights into material behavior and enables novel applications. While it is well-known that fitting multi-lobe BRDFs may be ill-posed under certain conditions, prior to this work, precise results for the well-posedness of BRDF estimation had remained elusive. Since our BRDF representation is derived from physical intuition, but relies on data, we avoid pitfalls of both parametric (low generalizability) and non-parametric regression (low interpretability, curse of dimensionality). Finally, we discuss several applications such as single-image relighting, light source estimation and physically meaningful BRDF editing.

[1]  Wojciech Matusik,et al.  Inverse shade trees for non-parametric material representation and editing , 2006, ACM Trans. Graph..

[2]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[3]  P. Diaconis,et al.  On Nonlinear Functions of Linear Combinations , 1984 .

[4]  Richard M. Everson,et al.  Independent Components Analysis , 2000, Artificial Neural Networks in Biomedicine.

[5]  W. Cleveland Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[6]  André Gagalowicz,et al.  Image-based rendering of diffuse, specular and glossy surfaces from a single image , 2001, SIGGRAPH.

[7]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[8]  Michael D. McCool,et al.  Homomorphic factorization of BRDFs for high-performance rendering , 2001, SIGGRAPH.

[9]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[10]  R. E. Carlson,et al.  Monotone Piecewise Cubic Interpolation , 1980 .

[11]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[12]  Ko Nishino Directional statistics BRDF model , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[14]  Todd E. Zickler,et al.  Blind Reflectometry , 2010, ECCV.

[15]  Donald P. Greenberg,et al.  Non-linear approximation of reflectance functions , 1997, SIGGRAPH.

[16]  Christina Gloeckner,et al.  Modern Applied Statistics With S , 2003 .

[17]  Frédo Durand,et al.  Experimental analysis of BRDF models , 2005, EGSR '05.

[18]  James F. Blinn,et al.  Texture and reflection in computer generated images , 1998 .

[19]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[20]  J. Horowitz Semiparametric Methods in Econometrics , 2011 .

[21]  P. Hanrahan,et al.  Light Scattering from Human Hair Fibers , 2003 .

[22]  M. Yuan ON THE IDENTIFIABILITY OF ADDITIVE INDEX MODELS , 2011 .

[23]  Sang Wook Lee,et al.  Global optimization for estimating a BRDF with multiple specular lobes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  G. Wolberg,et al.  An energy-minimization framework for monotonic cubic spline interpolation , 2002 .