Face relighting based on virtual irradiance sphere and reflection coefficient

We present a novel method to estimate the light source direction and relight a face texture image of a single 3-D model under arbitrary unknown illumination conditions. We create a virtual irradiance sphere to detect the light source direction from a given illuminated texture image using both normal vector mapping and weighted bilinear interpolation. We then induce a relighting equation with estimated ambient and diffuse intensity. We provide the result of a series of experiments on light source estimation, relighting, and face recognition to show the efficiency and accuracy of the proposed method in restoring the shading and shadow areas of a face texture image. Our approach for face relighting can be used for illuminant-invariant face recognition applications.

[1]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[2]  Yang Wang,et al.  Estimation of multiple directional light sources for synthesis of augmented reality images , 2002, Graph. Model..

[3]  M. Brooks,et al.  Revisiting Pentland's estimator of light source direction , 1994 .

[4]  Michael C. Loui.  Weighted derivation trees , 1976, CACM.

[5]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[7]  Yee-Hong Yang,et al.  Multiple Illuminant Direction Detection with Application to Image Synthesis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[10]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[11]  Zicheng Liu,et al.  Face relighting with radiance environment maps , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  Mike Brookes,et al.  Multiple light source detection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Chris Boehnen,et al.  3D Face Recognition Using 3D Alignment for PCA , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[14]  Katsushi Ikeuchi,et al.  Multiple light sources and reflectance property estimation based on a mixture of spherical distributions , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Yee-Hong Yang,et al.  Illuminant direction determination for multiple light sources , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  Takahiro Okabe,et al.  Spherical harmonics vs. Haar wavelets: basis for recovering illumination from cast shadows , 2004, CVPR 2004.

[18]  Gaile G. Gordon,et al.  Face recognition based on depth maps and surface curvature , 1991, Optics & Photonics.

[19]  Lei Zhang,et al.  Face synthesis and recognition from a single image under arbitrary unknown lighting using a spherical harmonic basis morphable model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Raghu Machiraju,et al.  Estimation of 3D faces and illumination from single photographs using a bilinear illumination model , 2005, EGSR '05.

[21]  A. Pentland.  Finding the illuminant direction , 1982 .