Measurement‐Based Synthesis of Facial Microgeometry

We present a technique for generating microstructure‐level facial geometry by augmenting a mesostructure‐level facial scan with detail synthesized from a set of exemplar skin patches scanned at much higher resolution. Additionally, we make point‐source reflectance measurements of the skin patches to characterize the specular reflectance lobes at this smaller scale and analyze facial reflectance variation at both the mesostructure and microstructure scales. We digitize the exemplar patches with a polarization‐based computational illumination technique which considers specular reflection and single scattering. The recorded microstructure patches can be used to synthesize full‐facial microstructure detail for either the same subject or to a different subject. We show that the technique allows for greater realism in facial renderings including more accurate reproduction of skin's specular reflection effects.

[1]  Stephen H. Westin,et al.  Measuring and modeling the appearance of finished wood , 2005, ACM Trans. Graph..

[2]  Ralf Sarlette,et al.  Efficient Resampling, Compression and Rendering of Metallic and Pearlescent Paint , 2009, VMV.

[3]  Sylvain Lefebvre,et al.  State of the Art in Example-based Texture Synthesis , 2009, Eurographics.

[4]  Kristin J. Dana,et al.  Skin Texture Modeling , 2005, International Journal of Computer Vision.

[5]  Eugene d'Eon,et al.  A quantized-diffusion model for rendering translucent materials , 2011, ACM Trans. Graph..

[6]  Thomas Malzbender,et al.  Polynomial texture maps , 2001, SIGGRAPH.

[7]  Ravi Ramamoorthi,et al.  Reflectance sharing: predicting appearance from a sparse set of images of a known shape , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[9]  Edward H. Adelson,et al.  Microgeometry capture using an elastomeric sensor , 2011, ACM Trans. Graph..

[10]  Gabriel Taubin,et al.  Appying Shape from Lighting Variation to Bump Map Capture , 1997, Rendering Techniques.

[11]  Jan J. Koenderink,et al.  The secret of velvety skin , 2003, Machine Vision and Applications.

[12]  Steve Marschner,et al.  Image-Based BRDF Measurement Including Human Skin , 1999, Rendering Techniques.

[13]  Henning Biermann,et al.  Texture and Shape Synthesis on Surfaces , 2001, Rendering Techniques.

[14]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[15]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[16]  Stephen Lin,et al.  Multiresolution reflectance filtering , 2005, EGSR '05.

[17]  Edward H. Adelson,et al.  Microgeometry capture using an elastomeric sensor , 2011, SIGGRAPH 2011.

[18]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[19]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[20]  Baining Guo,et al.  Synthesis of bidirectional texture functions on arbitrary surfaces , 2002, SIGGRAPH.

[21]  Robert J. Woodham,et al.  Photometric Stereo: A Reflectance Map Technique For Determining Surface Orientation From Image Intensity , 1979, Optics & Photonics.

[22]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[23]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[24]  Gino Acevedo,et al.  An accurate method for acquiring high resolution skin displacement maps , 2010, SIGGRAPH '10.

[25]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[26]  Jessica C. Ramella-Roman,et al.  Out of Plane Polarimetric Imaging of Skin: Surface and Subsurface Effect , 2008 .

[27]  Pieter Peers,et al.  Practical modeling and acquisition of layered facial reflectance , 2008, SIGGRAPH Asia '08.

[28]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  Peter Shirley,et al.  A microfacet-based BRDF generator , 2000, SIGGRAPH.

[30]  S. Marschner,et al.  Measuring and modeling the appearance of finished wood , 2005, SIGGRAPH 2005.

[31]  Paul E. Debevec,et al.  Multiview face capture using polarized spherical gradient illumination , 2011, ACM Trans. Graph..

[32]  Wojciech Matusik,et al.  A statistical model for synthesis of detailed facial geometry , 2006, ACM Trans. Graph..

[33]  R. Ramamoorthi,et al.  Frequency domain normal map filtering , 2007, SIGGRAPH 2007.

[34]  M. Gross,et al.  Analysis of human faces using a measurement-based skin reflectance model , 2006, ACM Trans. Graph..

[35]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[36]  Zicheng Liu,et al.  Image-based surface detail transfer , 2004, IEEE Computer Graphics and Applications.

[37]  Thabo Beeler,et al.  High-quality single-shot capture of facial geometry , 2010, ACM Trans. Graph..

[38]  Klaus Mueller,et al.  Generating sub-resolution detail in images and volumes using constrained texture synthesis , 2004, IEEE Visualization 2004.

[39]  Kavita Bala,et al.  Constrained Texture Synthesis via Energy Minimization , 2007, IEEE Transactions on Visualization and Computer Graphics.

[40]  Sylvain Lefebvre,et al.  Appearance-space texture synthesis , 2006, ACM Trans. Graph..

[41]  Pieter Peers,et al.  Temporal upsampling of performance geometry using photometric alignment , 2010, TOGS.

[42]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[43]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[44]  Craig Donner,et al.  Light diffusion in multi-layered translucent materials , 2005, SIGGRAPH 2005.

[45]  Steve Marschner,et al.  Building volumetric appearance models of fabric using micro CT imaging , 2011, ACM Trans. Graph..

[46]  Norimichi Tsumura,et al.  Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin , 2003, ACM Trans. Graph..

[47]  Marc Levoy,et al.  Texture synthesis over arbitrary manifold surfaces , 2001, SIGGRAPH.

[48]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[49]  John M. Snyder,et al.  Manifold bootstrapping for SVBRDF capture , 2010, ACM Trans. Graph..

[50]  Pat Hanrahan,et al.  Reflection from layered surfaces due to subsurface scattering , 1993, SIGGRAPH.

[51]  Sylvain Lefebvre,et al.  Parallel controllable texture synthesis , 2005, ACM Trans. Graph..