A Gaussian Process Latent Variable Model for BRDF Inference

The problem of estimating a full BRDF from partial observations has already been studied using either parametric or non-parametric approaches. The goal in each case is to best match this sparse set of input measurements. In this paper we address the problem of inferring higher order reflectance information starting from the minimal input of a single BRDF slice. We begin from the prototypical case of a homogeneous sphere, lit by a head-on light source, which only holds information about less than 0.001% of the whole BRDF domain. We propose a novel method to infer the higher dimensional properties of the material's BRDF, based on the statistical distribution of known material characteristics observed in real-life samples. We evaluated our method based on a large set of experiments generated from real-world BRDFs and newly measured materials. Although inferring higher dimensional BRDFs from such modest training is not a trivial problem, our method performs better than state-of-the-art parametric, semi-parametric and non-parametric approaches. Finally, we discuss interesting applications on material re-lighting, and flash-based photography.

[1]  Maja Pantic,et al.  View-Constrained Latent Variable Model for Multi-view Facial Expression Classification , 2014, ISVC.

[2]  Roland W Fleming,et al.  Real-world illumination and the perception of surface reflectance properties. , 2003, Journal of vision.

[3]  André Gagalowicz,et al.  Image-based rendering of diffuse, specular and glossy surfaces from a single image , 2001, SIGGRAPH.

[4]  Tianli Yu,et al.  Sparse lumigraph relighting by illumination and reflectance estimation from multi-view images , 2006, SIGGRAPH '06.

[5]  Peter Shirley,et al.  An Anisotropic Phong BRDF Model , 2000, J. Graphics, GPU, & Game Tools.

[6]  Donald P. Greenberg,et al.  A comprehensive physical model for light reflection , 1991, SIGGRAPH.

[7]  Stephen H. Westin,et al.  Image-based bidirectional reflectance distribution function measurement. , 2000, Applied optics.

[8]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[9]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[10]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[11]  Ravi Ramamoorthi,et al.  What an image reveals about material reflectance , 2011, 2011 International Conference on Computer Vision.

[12]  S. Sundararajan,et al.  Predictive Approaches for Choosing Hyperparameters in Gaussian Processes , 1999, Neural Computation.

[13]  Wojciech Matusik,et al.  Inverse shade trees for non-parametric material representation and editing , 2006, ACM Trans. Graph..

[14]  René Vidal,et al.  Latent Space Sparse Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Michael H. Brill,et al.  Color appearance models , 1998 .

[16]  E. Adelson,et al.  Image statistics for surface reflectance perception. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[17]  Mark D. Fairchild,et al.  Color Appearance Models , 1997, Computer Vision, A Reference Guide.

[18]  Wojciech Matusik,et al.  Inverse shade trees for non-parametric material representation and editing , 2006, SIGGRAPH 2006.

[19]  Katsushi Ikeuchi,et al.  Determining Reflectance Parameters and Illumination Distribution from a Sparse Set of Images for View-dependent Image Synthesis , 2001, ICCV.

[20]  Luc Van Gool,et al.  Tackling Shapes and BRDFs Head-On , 2014, 2014 2nd International Conference on 3D Vision.

[21]  Ko Nishino,et al.  Reflectance and Natural Illumination from a Single Image , 2012, ECCV.

[22]  Didier Stricker,et al.  Robust and Accurate Non-parametric Estimation of Reflectance Using Basis Decomposition and Correction Functions , 2014, ECCV.

[23]  Ron O Dror,et al.  Statistical characterization of real-world illumination. , 2004, Journal of vision.

[24]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[25]  Christophe Schlick,et al.  An Inexpensive BRDF Model for Physically‐based Rendering , 1994, Comput. Graph. Forum.

[26]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[27]  Rajesh P. N. Rao,et al.  Learning Shared Latent Structure for Image Synthesis and Robotic Imitation , 2005, NIPS.

[28]  Jason Lawrence,et al.  A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance , 2010, ACM Transactions on Graphics.

[29]  Katsushi Ikeuchi,et al.  Mixture of Spherical Distributions for Single-View Relighting , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  J. Koenderink,et al.  Phenomenological description of bidirectional surface reflection , 1998 .

[31]  Donald P. Greenberg,et al.  Toward a psychophysically-based light reflection model for image synthesis , 2000, SIGGRAPH.

[32]  F. E. Nicodemus Directional Reflectance and Emissivity of an Opaque Surface , 1965 .

[33]  Szymon Rusinkiewicz,et al.  A New Change of Variables for Efficient BRDF Representation , 1998, Rendering Techniques.

[34]  David J. Kriegman,et al.  Toward a perceptual space for gloss , 2009, TOGS.

[35]  Todd E. Zickler,et al.  Blind Reflectometry , 2010, ECCV.

[36]  Ko Nishino Directional statistics BRDF model , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[37]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[38]  James Arvo,et al.  Barycentric parameterizations for isotropic BRDFs , 2005, IEEE Transactions on Visualization and Computer Graphics.

[39]  Donald P. Greenberg,et al.  Non-linear approximation of reflectance functions , 1997, SIGGRAPH.

[40]  Maja Pantic,et al.  Discriminative Shared Gaussian Processes for Multiview and View-Invariant Facial Expression Recognition , 2015, IEEE Transactions on Image Processing.

[41]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[42]  M. Ashikhmin,et al.  Distribution-based BRDFs , 2007 .

[43]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[44]  知子 松井,et al.  Shared Gaussian Process Latent Variable Modelによるピアノ楽曲の自動採譜 , 2017 .

[45]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[46]  Frédo Durand,et al.  Experimental analysis of BRDF models , 2005, EGSR '05.