Recovering 3-D Shape and Reflectance From a Small Number of Photographs

There are computer graphics applications for which the shape and reflectance of complex objects, such as faces, cannot be obtained using specialized equipment due to cost and practical considerations. We present an image based technique that uses only a small number of example images, and assumes a parametric model of reflectance, to simultaneously and reliably recover the Bidirectional Reflectance Distributions Function (BRDF) and the 3-D shape of non-Lambertian objects. No information about the position and intensity of the light-sources or the position of the camera is required. We successfully apply this approach to human faces, accurately recovering their 3-D shape and BRDF. We use the recovered information to efficiently and accurately render photorealistic images of the faces under novel illumination conditions in which the rendered image intensity closely matches the intensity in real images. The accuracy of our technique is further demonstrated by the close resemblance of the skin BRDF recovered using our method, to the one measured with a method presented in the literature and in which a 3-D scanner was used.

[1]  F. E. Nicodemus,et al.  Geometrical considerations and nomenclature for reflectance , 1977 .

[2]  Kosuke Sato,et al.  Determining Reflectance Properties of an Object Using Range and Brightness Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Robert J. Woodham,et al.  Analysing Images of Curved Surfaces , 1981, Artif. Intell..

[5]  Lawrence B. Wolff,et al.  Surface Curvature and Shape Reconstruction from Unknown Multiple Illumination and Integrability , 1997, Comput. Vis. Image Underst..

[6]  J. Koenderink,et al.  Optical properties (bidirectional reflection distribution functions) of velvet. , 1998, Applied optics.

[7]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[8]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[9]  H. Helmholtz Helmholtz's Treatise on Physiological Optics , 1963 .

[10]  Daniel Snow,et al.  Shape and albedo from multiple images using integrability , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  H.J.C.M. Sterenborg,et al.  Skin optics , 1989, IEEE Transactions on Biomedical Engineering.

[12]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[13]  Shree K. Nayar,et al.  Generalization of the Lambertian model and implications for machine vision , 1995, International Journal of Computer Vision.

[14]  Bui Tuong Phong Illumination for computer generated pictures , 1975, Commun. ACM.

[15]  Rui J. P. de Figueiredo,et al.  A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Katsushi Ikeuchi,et al.  Extracting the Shape and Roughness of Specular Lobe Objects Using Four Light Photometric Stereo , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[18]  M. Pinar Mengüç,et al.  Thermal Radiation Heat Transfer , 2020 .

[19]  Steve Marschner,et al.  Image-Based BRDF Measurement Including Human Skin , 1999, Rendering Techniques.

[20]  Shirley Dex,et al.  JR 旅客販売総合システム(マルス)における運用及び管理について , 1991 .

[21]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[22]  Ondrej Drbohlav,et al.  Specularities Reduce Ambiguity of Uncalibrated Photometric Stereo , 2002, ECCV.

[23]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Pat Hanrahan,et al.  Reflection from layered surfaces due to subsurface scattering , 1993, SIGGRAPH.

[25]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[26]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[27]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[28]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[29]  Sang Wook Lee,et al.  Estimation of diffuse and specular appearance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[31]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .