Image based surface reflectance remapping for consistent and tool independent material appearence

Physically-based rendering in Computer Graphics requires the knowledge of material properties other than 3D shapes, textures and colors, in order to solve the rendering equation. A number of material models have been developed, since no model is currently able to reproduce the full range of available materials. Although only few material models have been widely adopted in current rendering systems, the lack of standardisation causes several issues in the 3D modelling workflow, leading to a heavy tool dependency of material appearance. In industry, final decisions about products are often based on a virtual prototype, a crucial step for the production pipeline, usually developed by a collaborations among several departments, which exchange data. Unfortunately, exchanged data often tends to differ from the original, when imported into a different application. As a result, delivering consistent visual results requires time, labour and computational cost. This thesis begins with an examination of the current state of the art in material appearance representation and capture, in order to identify a suitable strategy to tackle material appearance consistency. Automatic solutions to this problem are suggested in this work, accounting for the constraints of real-world scenarios, where the only available information is a reference rendering and the renderer used to obtain it, with no access to the implementation of the shaders. In particular, two image-based frameworks are proposed, working under these constraints. The first one, validated by means of perceptual studies, is aimed to the remapping of BRDF parameters and useful when the parameters used for the reference rendering are available. The second one provides consistent material appearance across different renderers, even when the parameters used for the reference are unknown. It allows the selection of an arbitrary reference rendering tool, and manipulates the output of other renderers in order to be consistent with the reference.

[1]  Barton L Anderson,et al.  Image statistics and the perception of surface gloss and lightness. , 2010, Journal of vision.

[2]  Wojciech Matusik,et al.  Inverse shade trees for non-parametric material representation and editing , 2006, ACM Trans. Graph..

[3]  Paul Lalonde,et al.  A Wavelet Representation of Reflectance Functions , 1997, IEEE Trans. Vis. Comput. Graph..

[4]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[5]  Athinodoros S. Georghiades,et al.  Recovering 3-D Shape and Reflectance From a Small Number of Photographs , 2003, Rendering Techniques.

[6]  Sabrina Hansmann-Roth,et al.  The dress and individual differences in the perception of surface properties , 2017, Vision Research.

[7]  Jaakko Lehtinen,et al.  Practical SVBRDF capture in the frequency domain , 2013, ACM Trans. Graph..

[8]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Christophe Schlick,et al.  An Inexpensive BRDF Model for Physically‐based Rendering , 1994, Comput. Graph. Forum.

[10]  Ramesh Raskar,et al.  Single view reflectance capture using multiplexed scattering and time-of-flight imaging , 2011, SA '11.

[11]  Min H. Kim,et al.  Characterization for High Dynamic Range Imaging , 2008, Comput. Graph. Forum.

[12]  Karol Myszkowski,et al.  Rendering Pearlescent Appearance Based On Paint‐Composition Modelling , 2001, Comput. Graph. Forum.

[13]  Steve Marschner,et al.  Discrete stochastic microfacet models , 2014, ACM Trans. Graph..

[14]  Baining Guo,et al.  Pocket reflectometry , 2011, SIGGRAPH 2011.

[15]  T. Trowbridge,et al.  Average irregularity representation of a rough surface for ray reflection , 1975 .

[16]  Hans-Peter Seidel,et al.  An intuitive control space for material appearance , 2016, ACM Trans. Graph..

[17]  Mark D. Fairchild,et al.  Color Appearance Models , 1997, Computer Vision, A Reference Guide.

[18]  Barton L Anderson,et al.  Image statistics do not explain the perception of gloss and lightness. , 2009, Journal of vision.

[19]  Pat Hanrahan,et al.  Reflection from layered surfaces due to subsurface scattering , 1993, SIGGRAPH.

[20]  Eugene L. Church,et al.  Optimal estimation of finish parameters , 1991, Optics & Photonics.

[21]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[22]  Paul Graham,et al.  Acquiring reflectance and shape from continuous spherical harmonic illumination , 2013, ACM Trans. Graph..

[23]  Maria L. Rizzo,et al.  Measuring and testing dependence by correlation of distances , 2007, 0803.4101.

[24]  Frédo Durand,et al.  Experimental analysis of BRDF models , 2005, EGSR '05.

[25]  E. Heitz.  Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs , 2014 .

[26]  Scott Onstott.  Adobe Photoshop CS6 Essentials , 2012 .

[27]  Jaakko Lehtinen,et al.  Two-shot SVBRDF capture for stationary materials , 2015, ACM Trans. Graph..

[28]  Pascal Barla,et al.  In Praise of an Alternative BRDF Parametrization , 2015, Material Appearance Modeling.

[29]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[30]  Eugene d'Eon,et al.  A quantized-diffusion model for rendering translucent materials , 2011, ACM Trans. Graph..

[31]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[32]  R. Fleming.  Visual perception of materials and their properties , 2014, Vision Research.

[33]  Murat Kurt,et al.  A survey of BRDF models for computer graphics , 2009, COMG.

[34]  Tim Weyrich,et al.  Principles of Appearance Acquisition and Representation , 2009, Found. Trends Comput. Graph. Vis..

[35]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[36]  H. Jensen,et al.  Wavelet importance sampling: efficiently evaluating products of complex functions , 2005, SIGGRAPH 2005.

[37]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  F. Clarke,et al.  Helmholtz Reciprocity: its validity and application to reflectometry , 1985 .

[39]  Karol Myszkowski,et al.  Reverse engineering approach to appearance-based design of metallic and pearlescent paints , 2004, The Visual Computer.

[40]  James T. Kajiya,et al.  Rendering fur with three dimensional textures , 1989, SIGGRAPH.

[41]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[42]  James Arvo,et al.  Applications of irradiance tensors to the simulation of non-Lambertian phenomena , 1995, SIGGRAPH.

[43]  Giuseppe Claudio Guarnera,et al.  Capturing and representing brdfs for virtual reality , 2016, SIGGRAPH ASIA Courses.

[44]  Barton L Anderson,et al.  Visual perception of materials and surfaces , 2011, Current Biology.

[45]  Yannick Boucher,et al.  Wavelet-based modeling of spectral bidirectional reflectance distribution function data , 2004 .

[46]  Brian E. Smits,et al.  Practical physically-based shading in film and game production , 2012, SIGGRAPH '12.

[47]  Hans-Peter Seidel,et al.  Mesostructure from Specularity , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[48]  Tim Weyrich,et al.  Image-based Remapping of Material Appearance , 2017, MAM@EGSR.

[49]  Hans-Peter Seidel,et al.  DISCO: acquisition of translucent objects , 2004, ACM Trans. Graph..

[50]  Nelson L. Max,et al.  Bidirectional reflection functions from surface bump maps , 1987, SIGGRAPH.

[51]  Jack J. Hsia,et al.  Bidirectional Reflectometry. Part I. , 1976, Journal of research of the National Bureau of Standards. Section A, Physics and chemistry.

[52]  Stephen Lin,et al.  Photorealistic rendering of knitwear using the lumislice , 2001, SIGGRAPH.

[53]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[54]  Nicolas Bonneel,et al.  A Practical Framework for Sharing and Rendering Real-World Bidirectional Scattering Distribution Functions , 2014 .

[55]  Holly Rushmeier,et al.  Experiments with a low-cost system for computer graphics material model acquisition , 2015, Electronic Imaging.

[56]  Pieter Peers,et al.  Practical modeling and acquisition of layered facial reflectance , 2008, SIGGRAPH Asia '08.

[57]  Tomas Akenine-Möller,et al.  Real-time rendering, 3rd Edition , 2008 .

[58]  Steve Marschner,et al.  Image-Based BRDF Measurement Including Human Skin , 1999, Rendering Techniques.

[59]  Peter Shirley,et al.  A microfacet-based BRDF generator , 2000, SIGGRAPH.

[60]  Peter Shirley,et al.  The halfway vector disk for BRDF modeling , 2006, TOGS.

[61]  Yasushi Yagi,et al.  Multiplexed Illumination for Measuring BRDF Using an Ellipsoidal Mirror and a Projector , 2007, ACCV.

[62]  David Geisler-Moroder,et al.  A New Ward BRDF Model with Bounded Albedo , 2010, Comput. Graph. Forum.

[63]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[64]  B. Smith,et al.  Geometrical shadowing of a random rough surface , 1967 .

[65]  László Szirmay-Kalos,et al.  Compact Metallic Reflectance Models , 1999, Comput. Graph. Forum.

[66]  Steven M. Seitz,et al.  Shape and Spatially-Varying BRDFs from Photometric Stereo , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Pieter Peers,et al.  Estimating Surface Normals from Spherical Stokes Reflectance Fields , 2012, ECCV Workshops.

[68]  Pieter Peers,et al.  genBRDF: discovering new analytic BRDFs with genetic programming , 2014, ACM Trans. Graph..

[69]  Stephen H. Westin,et al.  A Comparison of Four BRDF Models , 2005 .

[70]  Pieter Peers,et al.  Estimating Specular Roughness and Anisotropy from Second Order Spherical Gradient Illumination , 2009, Comput. Graph. Forum.

[71]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  Donald P. Greenberg,et al.  Non-linear approximation of reflectance functions , 1997, SIGGRAPH.

[73]  Philippe Bekaert,et al.  High quality mesostructure acquisition using specularities , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Arne Dür,et al.  An Improved Normalization for the Ward Reflectance Model , 2006, J. Graph. Tools.

[75]  Szymon Rusinkiewicz,et al.  Efficient BRDF importance sampling using a factored representation , 2004, SIGGRAPH 2004.

[76]  Giuseppe Claudio Guarnera,et al.  Material capture and representation with applications in virtual reality , 2017, SIGGRAPH Courses.

[77]  Julie Dorsey,et al.  Digital Modeling of Material Appearance , 2007 .

[78]  John M. Snyder,et al.  Manifold bootstrapping for SVBRDF capture , 2010, ACM Trans. Graph..

[79]  Alexander Keller,et al.  The Material Definition Language , 2015, Material Appearance Modeling.

[80]  László Szirmay-Kalos,et al.  An anisotropic BRDF model for fitting and Monte Carlo rendering , 2010, COMG.

[81]  Jan Kautz,et al.  Interactive rendering with arbitrary BRDFs using separable approximations , 1999, SIGGRAPH '99.

[82]  Alexander Wilkie,et al.  Arbitrarily layered micro-facet surfaces , 2007, GRAPHITE '07.

[83]  Nicolas Holzschuch,et al.  Accurate fitting of measured reflectances using a Shifted Gamma micro‐facet distribution , 2012, Comput. Graph. Forum.

[84]  Caroline Jay,et al.  A perceptually validated model for surface depth hallucination , 2008, ACM Trans. Graph..

[85]  Alain Fournier,et al.  Separating Reflection Functions for Linear Radiosity , 1995, Rendering Techniques.

[86]  Pierre Poulin,et al.  Rational BRDF , 2012, IEEE Transactions on Visualization and Computer Graphics.

[87]  J. Krauskopf,et al.  Color discrimination and adaptation , 1992, Vision Research.

[88]  Andrew Gardner,et al.  Linear light source reflectometry , 2003, ACM Trans. Graph..

[89]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[90]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[91]  M. Ashikhmin,et al.  Distribution-based BRDFs , 2007 .

[92]  Steve Marschner,et al.  Matching Real Fabrics with Micro-Appearance Models , 2015, ACM Trans. Graph..

[93]  Giuseppe Claudio Guarnera,et al.  BRDF Representation and Acquisition , 2016, Comput. Graph. Forum.

[94]  Pieter Peers,et al.  Mobile Surface Reflectometry , 2016, Comput. Graph. Forum.

[95]  Giuseppe Claudio Guarnera,et al.  Towards a consistent, tool independent virtual material appearance , 2017 .

[96]  Peter Shirley,et al.  An Anisotropic Phong BRDF Model , 2000, J. Graphics, GPU, & Game Tools.

[97]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[98]  Donald P. Greenberg,et al.  Modeling the interaction of light between diffuse surfaces , 1984, SIGGRAPH.

[99]  Ravi Ramamoorthi,et al.  Reflectance sharing: predicting appearance from a sparse set of images of a known shape , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[100]  Pascal Barla,et al.  Statistical analysis of bidirectional reflectance distribution functions , 2015, Electronic Imaging.

[101]  Stephen H. Westin,et al.  Predicting reflectance functions from complex surfaces , 1992, SIGGRAPH.

[102]  Frédo Durand,et al.  Data-driven hallucination of different times of day from a single outdoor photo , 2013, ACM Trans. Graph..

[103]  K. Bala,et al.  A radiative transfer framework for rendering materials with anisotropic structure , 2010, SIGGRAPH 2010.

[104]  Derek Bradley,et al.  Recent Advances in Facial Appearance Capture , 2015, Comput. Graph. Forum.

[105]  Steve Marschner,et al.  Microfacet Models for Refraction through Rough Surfaces , 2007, Rendering Techniques.

[106]  Mathias Paulin,et al.  BRDF Measurement Modelling using Wavelets for Efficient Path Tracing , 2003, Comput. Graph. Forum.

[107]  Marc Stamminger,et al.  Translucent Shadow Maps , 2003, Rendering Techniques.

[108]  Pieter Peers,et al.  Reflectance scanning , 2014, ACM Trans. Graph..

[109]  Ralf Sarlette,et al.  Photo‐realistic Rendering of Metallic Car Paint from Image‐Based Measurements , 2008, Comput. Graph. Forum.

[110]  Steven A. Cholewiak,et al.  Towards a unified explanation of shape from shading and texture , 2013 .

[111]  G. Wyszecki,et al.  Color Science Concepts and Methods , 1982 .

[112]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[113]  Amitabh Varshney,et al.  Interactive subsurface scattering for translucent meshes , 2003, I3D '03.

[114]  Hans-Peter Seidel,et al.  Validation of Color Managed 3D Appearance Acquisition , 2004, Color Imaging Conference.

[115]  Carsten Dachsbacher,et al.  The SGGX microflake distribution , 2015, ACM Trans. Graph..

[116]  Matteo Valsecchi,et al.  Lightness perception for matte and glossy complex shapes , 2017, Vision Research.

[117]  Pierre Poulin,et al.  Extracting Microfacet‐based BRDF Parameters from Arbitrary Materials with Power Iterations , 2015, Comput. Graph. Forum.

[118]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[119]  Ken Perlin,et al.  Measuring bidirectional texture reflectance with a kaleidoscope , 2003, ACM Trans. Graph..

[120]  Rafal Mantiuk,et al.  Comparison of Four Subjective Methods for Image Quality Assessment , 2012, Comput. Graph. Forum.

[121]  Mashhuda Glencross,et al.  A case study evaluation: perceptually accurate textured surface models , 2009, APGV '09.

[122]  Wojciech Matusik,et al.  Efficient Isotropic BRDF Measurement , 2003, Rendering Techniques.

[123]  Michael D. McCool,et al.  Homomorphic factorization of BRDFs for high-performance rendering , 2001, SIGGRAPH.

[124]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[125]  Nicolas Bonneel,et al.  Reducing Anisotropic BSDF Measurement to Common Practice , 2014, Material Appearance Modeling.

[126]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[127]  Robert R. Lewis,et al.  Making Shaders More Physically Plausible , 1994, Comput. Graph. Forum.

[128]  Murat Kurt,et al.  A General BRDF Representation Based on Tensor Decomposition , 2011, Comput. Graph. Forum.

[129]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[130]  Donald P. Greenberg,et al.  A comprehensive physical model for light reflection , 1991, SIGGRAPH.

[131]  Thorsten Grosch,et al.  Consistent Scene Editing by Progressive Difference Images , 2015, Comput. Graph. Forum.

[132]  Shree K. Nayar,et al.  Generalization of Lambert's reflectance model , 1994, SIGGRAPH.

[133]  Moshe Ben-Ezra,et al.  An LED-only BRDF measurement device , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[134]  E. L. Church,et al.  The Prediction Of BRDFs From Surface Profile Measurements , 1990, Optics & Photonics.

[135]  Leonidas J. Guibas,et al.  Robust Monte Carlo methods for light transport simulation , 1997 .

[136]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[137]  Greg Ward,et al.  Picture Perfect RGB Rendering Using Spectral Prefiltering and Sharp Color Primaries , 2002, Rendering Techniques.

[138]  James Arvo,et al.  Barycentric parameterizations for isotropic BRDFs , 2005, IEEE Transactions on Visualization and Computer Graphics.

[139]  Anders Ynnerman,et al.  BRDF models for accurate and efficient rendering of glossy surfaces , 2012, TOGS.

[140]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[141]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[142]  Steve Marschner,et al.  Building volumetric appearance models of fabric using micro CT imaging , 2014, Commun. ACM.

[143]  Jason Lawrence,et al.  A photometric approach for estimating normals and tangents , 2008, ACM Trans. Graph..

[144]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[145]  Giuseppe Claudio Guarnera,et al.  Absolute colorimetric characterization of a DSLR camera , 2014, Electronic Imaging.

[146]  Ivan E. Sutherland,et al.  The Ultimate Display , 1965 .

[147]  Ko Nishino,et al.  Directional statistics-based reflectance model for isotropic bidirectional reflectance distribution functions. , 2011, Journal of the Optical Society of America. A, Optics, image science, and vision.

[148]  Murat Kurt,et al.  Linear approximation of Bidirectional Reflectance Distribution Functions , 2008, Comput. Graph..

[149]  Pascal Barla,et al.  MatCap Decomposition for Dynamic Appearance Manipulation , 2015, EGSR.

[150]  Abhijeet Ghosh,et al.  Circularly polarized spherical illumination reflectometry , 2010, SIGGRAPH 2010.

[151]  Eric P. Lafortune,et al.  Using the modified Phong reflectance model for physically based rendering , 1994 .

[152]  Steve Marschner,et al.  Predicting Appearance from Measured Microgeometry of Metal Surfaces , 2015, ACM Trans. Graph..

[153]  Steve Marschner,et al.  Estimating dual-scale properties of glossy surfaces from step-edge lighting , 2011, ACM Trans. Graph..

[154]  Szymon Rusinkiewicz,et al.  A New Change of Variables for Efficient BRDF Representation , 1998, Rendering Techniques.

[155]  Noah Snavely,et al.  Material recognition in the wild with the Materials in Context Database , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[156]  Todd E. Zickler,et al.  Passive Reflectometry , 2008, ECCV.

[157]  Stephen H. Westin,et al.  Automated three-axis gonioreflectometer for computer graphics applications , 2005, SPIE Optics + Photonics.

[158]  Michael Frankfurter,et al.  Numerical Recipes In C The Art Of Scientific Computing , 2016 .