Acquiring the reflectance field of a human face

We present a method to acquire the reflectance field of a human face and use these measurements to render the face under arbitrary changes in lighting and viewpoint. We first acquire images of the face from a small set of viewpoints under a dense sampling of incident illumination directions using a light stage. We then construct a reflectance function image for each observed image pixel from its values over the space of illumination directions. From the reflectance functions, we can directly generate images of the face from the original viewpoints in any form of sampled or computed illumination. To change the viewpoint, we use a model of skin reflectance to estimate the appearance of the reflectance functions for novel viewpoints. We demonstrate the technique with synthetic renderings of a person's face under novel illumination and viewpoints.

[1]  I. W. Busbridge.  The Mathematics of Radiative Transfer. , 1960 .

[2]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[3]  Frederick I. Parke,et al.  Computer generated animation of faces , 1972, ACM Annual Conference.

[4]  F. E. Nicodemus,et al.  Geometrical considerations and nomenclature for reflectance , 1977 .

[5]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[6]  H.J.C.M. Sterenborg,et al.  Skin optics , 1989, IEEE Transactions on Biomedical Engineering.

[7]  Demetri Terzopoulos,et al.  Physically-based facial modelling, analysis, and animation , 1990, Comput. Animat. Virtual Worlds.

[8]  Lance Williams,et al.  Performance-driven facial animation , 1990, SIGGRAPH.

[9]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[10]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[11]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[12]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1992, SIGGRAPH.

[13]  Pat Hanrahan,et al.  Reflection from layered surfaces due to subsurface scattering , 1993, SIGGRAPH.

[14]  Katsushi Ikeuchi,et al.  Temporal-color space analysis of reflection , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Shree K. Nayar,et al.  Generalization of Lambert's reflectance model , 1994, SIGGRAPH.

[16]  Peter J. Hunter,et al.  A virtual environment and model of the eye for surgical simulation , 1994, SIGGRAPH.

[17]  Julie Dorsey,et al.  Effic ient Re-rendering of Naturally Illuminated Environments , 1994 .

[18]  Daniel Thalmann,et al.  A dynamic wrinkle model in facial animation and skin ageing , 1995, Comput. Animat. Virtual Worlds.

[19]  Demetri Terzopoulos,et al.  Realistic modeling for facial animation , 1995, SIGGRAPH.

[20]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[21]  Michael Gervautz,et al.  An Image based Measurement System for Anisotropic Reflection , 1996, Comput. Graph. Forum.

[22]  Lawrence A. Rowe,et al.  Compressed Domain Processing of JPEG-encoded imaages , 1996, Real Time Imaging.

[23]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[24]  Tien-Tsin Wong,et al.  Image-based Rendering with Controllable Illumination , 1997, Rendering Techniques.

[25]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[26]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[27]  Donald P. Greenberg,et al.  Non-linear approximation of reflectance functions , 1997, SIGGRAPH.

[28]  Shree K. Nayar,et al.  Reflectance and Texture of Real-World Surfaces Authors , 1997, CVPR 1997.

[29]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[30]  Gavin S. P. Miller,et al.  Lazy Decompression of Surface Light Fields for Precomputed Global Illumination , 1998, Rendering Techniques.

[31]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[32]  Frederick I. Parke,et al.  Computer gernerated animation of faces , 1998 .

[33]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[34]  Pascal Fua,et al.  From Regular Images to Animated Heads: A Least Squares Approach , 1998, ECCV.

[35]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[36]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[37]  Jan Kautz,et al.  Interactive rendering with arbitrary BRDFs using separable approximations , 1999, SIGGRAPH '99.

[38]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[39]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[40]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[41]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[42]  Steve Marschner,et al.  Image-Based BRDF Measurement Including Human Skin , 1999, Rendering Techniques.