Face Re-Lighting from a Single Image under Harsh Lighting Conditions

In this paper, we present a new method to change the illumination condition of a face image, with unknown face geometry and albedo information. This problem is particularly difficult when there is only one single image of the subject available and it was taken under a harsh lighting condition. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using spherical harmonic representation. However, the approximation error can be large under harsh lighting conditions thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion based framework that uses a Markov Random Field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to harsh lighting conditions, but insensitive to partial occlusions as well. The performance of our framework is demonstrated through various experimental results, including the improvement to the face recognition rate under harsh lighting conditions.

[1]  T. Kanade,et al.  Combining Models and Exemplars for Face Recognition: An Illuminating Example , 2001 .

[2]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[3]  Song-Chun Zhu,et al.  What are Textons? , 2005, Int. J. Comput. Vis..

[4]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Sami Romdhani,et al.  Efficient, robust and accurate fitting of a 3D morphable model , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[7]  Steve Marschner,et al.  Modeling and Rendering for Realistic Facial Animation , 2000, Rendering Techniques.

[8]  Steve Marschner,et al.  Image-Based BRDF Measurement Including Human Skin , 1999, Rendering Techniques.

[9]  Amnon Shashua,et al.  The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Pascal Fua,et al.  Recovery of Reflectances and Varying Illuminants from Multiple Views , 2002, ECCV.

[11]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.

[13]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[14]  Russell A. Epstein,et al.  5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[15]  Hans-Peter Seidel,et al.  Exchanging Faces in Images , 2004, Comput. Graph. Forum.

[16]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  James D. Foley,et al.  Fundamentals of interactive computer graphics , 1982 .

[19]  Raghu Machiraju,et al.  A bilinear illumination model for robust face recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[20]  Shuicheng Yan,et al.  Ranking prior likelihood distributions for Bayesian shape localization framework , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[22]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[23]  Lei Zhang,et al.  Face recognition under variable lighting using harmonic image exemplars , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[24]  Pascal Fua,et al.  Variable albedo surface reconstruction from stereo and shape from shading , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  Pascal Fua,et al.  Accurate face models from uncalibrated and ill-lit video sequences , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[26]  Sami Romdhani,et al.  Face identification across different poses and illuminations with a 3D morphable model , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[27]  Lei Zhang,et al.  Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Marc Olano,et al.  Reflection space image based rendering , 1999, SIGGRAPH.

[30]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[31]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  George Drettakis,et al.  Interactive Virtual Relighting of Real Scenes , 2000, IEEE Trans. Vis. Comput. Graph..

[33]  Pat Hanrahan,et al.  An efficient representation for irradiance environment maps , 2001, SIGGRAPH.

[34]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[35]  Lei Zhang,et al.  Face synthesis and recognition from a single image under arbitrary unknown lighting using a spherical harmonic basis morphable model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Erez Sali,et al.  Recognizing Novel 3-D Objects Under New Illumination and Viewing Position Using a Small Number of Examples. , 1998, ICCV 1998.

[37]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[38]  David J. Kriegman,et al.  Nine points of light: acquiring subspaces for face recognition under variable lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[40]  Arne Stoschek,et al.  Image-based re-rendering of faces for continuous pose and illumination directions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[41]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[42]  Zicheng Liu,et al.  Expressive expression mapping with ratio images , 2001, SIGGRAPH.

[43]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[44]  Zicheng Liu,et al.  Face relighting with radiance environment maps , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[45]  Rama Chellappa,et al.  Illumination-insensitive face recognition using symmetric shape-from-shading , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).