Image-based face illumination transferring using logarithmic total variation models

In this paper, we present a novel image-based technique that transfers illumination from a source face image to a target face image based on the Logarithmic Total Variation (LTV) model. Our method does not require any prior information regarding the lighting conditions or the 3D geometries of the underlying faces. We first use a Radial Basis Functions (RBFs)-based deformation technique to align key facial features of the reference 2D face with those of the target face. Then, we employ the LTV model to factorize each of the two aligned face images to an illumination-dependent component and an illumination-invariant component. Finally, illumination transferring is achieved by replacing the illumination-dependent component of the target face by that of the reference face. We tested our technique on numerous grayscale and color face images from various face datasets including the Yale face Database, as well as the application of illumination-preserved face coloring.

[1]  Gang Hua,et al.  Face Re-Lighting from a Single Image under Harsh Lighting Conditions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Arne Stoschek,et al.  Image-based re-rendering of faces for continuous pose and illumination directions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Jun-yong Noh,et al.  Animated deformations with radial basis functions , 2000, VRST '00.

[4]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[5]  Mila Nikolova,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004, Journal of Mathematical Imaging and Vision.

[6]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[7]  Jun-yong Noh,et al.  Expression cloning , 2001, SIGGRAPH.

[8]  Nira Dyn,et al.  Image Warping by Radial Basis Functions: Application to Facial Expressions , 1994, CVGIP Graph. Model. Image Process..

[9]  Zhigang Deng,et al.  Animating blendshape faces by cross-mapping motion capture data , 2006, I3D '06.

[10]  Wotao Yin,et al.  Parametric Maximum Flow Algorithms for Fast Total Variation Minimization , 2009, SIAM J. Sci. Comput..

[11]  Amnon Shashua,et al.  The quotient image: Class based recognition and synthesis under varying illumination conditions , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[12]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  A. Martínez,et al.  The AR face databasae , 1998 .

[14]  Wotao Yin,et al.  The Total Variation Regularized L1 Model for Multiscale Decomposition , 2007, Multiscale Model. Simul..

[15]  Zicheng Liu,et al.  Expressive expression mapping with ratio images , 2001, SIGGRAPH.

[16]  Zicheng Liu,et al.  Face relighting with radiance environment maps , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Steve Marschner,et al.  Inverse Lighting for Photography , 1997, CIC.

[18]  Dorin Comaniciu,et al.  Total variation models for variable lighting face recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  S. Osher,et al.  Total Variation Based Image Cartoon-Texture Decomposition , 2005 .

[20]  John P. Lewis,et al.  Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation , 2000, SIGGRAPH.

[21]  Gang Hua,et al.  Face Recognition using Discriminatively Trained Orthogonal Rank One Tensor Projections , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Pieter Peers,et al.  Post-Production Facial Performance Relighting Using , 2008 .

[23]  Yi Zhou,et al.  Bayesian tangent shape model: estimating shape and pose parameters via Bayesian inference , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..