Molding Face Shapes by Example

Human faces are remarkably similar in global properties, including size, aspect ratios, and locations of main features, but can vary considerably in details across individuals, gender, race, or due to facial expression. We propose a novel method for 3D shape recovery of a face from a single image using a single 3D reference model of a different person's face. The method uses the input image as a guide to mold the reference model to reach a desired reconstruction. Assuming Lambertian reflectance and rough alignment of the input image and reference model, we seek shape, albedo, and lighting that best fit the image while preserving the rough structure of the model. We demonstrate our method by providing accurate reconstructions of novel faces overcoming significant differences in shape due to gender, race, and facial expressions.

[1]  Michael Lindenbaum,et al.  Shape Reconstruction of 3D Bilaterally Symmetric Surfaces , 2000, International Journal of Computer Vision.

[2]  T. Kanade,et al.  Combining Models and Exemplars for Face Recognition: An Illuminating Example , 2001 .

[3]  Ronen Basri,et al.  Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting , 2004, ECCV.

[4]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  A. Pentland.  Finding the illuminant direction , 1982 .

[6]  Edwin R. Hancock,et al.  Recovering facial shape and albedo using a statistical model of surface normal direction , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[8]  P. Dupuis,et al.  An Optimal Control Formulation and Related Numerical Methods for a Problem in Shape Reconstruction , 1994 .

[9]  Rama Chellappa,et al.  Characterization of Human Faces under Illumination Variations Using Rank, Integrability, and Symmetry Constraints , 2004, ECCV.

[10]  Sami Romdhani,et al.  Efficient, robust and accurate fitting of a 3D morphable model , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Lei Zhang,et al.  Face recognition under variable lighting using harmonic image exemplars , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  R. Basri,et al.  Statistical Symmetric Shape from Shading for 3D Structure Recovery of Faces , 2004, eccv 2004.

[13]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[15]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Amnon Shashua,et al.  The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  M. Shah,et al.  Shape from shading with variable albedo , 1998 .

[18]  Rama Chellappa,et al.  Symmetric Shape-from-Shading Using Self-ratio Image , 2001, International Journal of Computer Vision.

[19]  Berthold K. P. Horn.  Obtaining shape from shading information , 1989 .

[20]  David J. Kriegman,et al.  From few to many: generative models for recognition under variable pose and illumination , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[21]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  S. Ullman,et al.  Generalization to Novel Images in Upright and Inverted Faces , 1993, Perception.

[23]  E. Rouy,et al.  A viscosity solutions approach to shape-from-shading , 1992 .

[24]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[25]  Ron Kimmel,et al.  Optimal Algorithm for Shape from Shading and Path Planning , 2001, Journal of Mathematical Imaging and Vision.

[26]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.