Face Recognition Based on Fitting a 3D Morphable Model

This paper presents a method for face recognition across variations in pose, ranging from frontal to profile views, and across a wide range of illuminations, including cast shadows and specular reflections. To account for these variations, the algorithm simulates the process of image formation in 3D space, using computer graphics, and it estimates 3D shape and texture of faces from single images. The estimate is achieved by fitting a statistical, morphable model of 3D faces to images. The model is learned from a set of textured 3D scans of heads. We describe the construction of the morphable model, an algorithm to fit the model to images, and a framework for face identification. In this framework, faces are represented by model parameters for 3D shape and texture. We present results obtained with 4,488 images from the publicly available CMU-PIE database and 1,940 images from the FERET database.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[3]  Edward H. Adelson,et al.  Merging Images Through Pattern Decomposition , 1985, Optics & Photonics.

[4]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[5]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Hiroshi Harashima,et al.  A system of analyzing and synthesizing facial images , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[7]  A. Kuijpers-Jagtman.  [Illuminating the face]. , 1993, Nederlands tijdschrift voor tandheelkunde.

[8]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[9]  Timothy F. Cootes,et al.  An Automatic Face Identification System Using Flexible Appearance Models , 1994, BMVC.

[10]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Christoph von der Malsburg,et al.  Single-View Based Recognition of Faces Rotated in Depth , 1995 .

[12]  Peter W. Hallinan,et al.  A deformable model for the recognition of human faces under arbitrary illumination , 1995 .

[13]  Emanuele Trucco,et al.  Computer and Robot Vision , 1995 .

[14]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[15]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[16]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[17]  Thomas S. Huang,et al.  3-D Face Modeling and Its Applications , 1996, Int. J. Pattern Recognit. Artif. Intell..

[18]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[19]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[20]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Timothy F. Cootes,et al.  Face Recognition Using Active Appearance Models , 1998, ECCV.

[22]  Thomas Vetter,et al.  Estimating Coloured 3D Face Models from Single Images: An Example Based Approach , 1998, ECCV.

[23]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[24]  D. B. Graham,et al.  Face recognition from unfamiliar views: subspace methods and pose dependency , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[25]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[26]  Volker Blanz,et al.  Automatische Rekonstruktion der dreidimensionalen Form von Gesichtern aus einem Einzelbild , 2000 .

[27]  Rama Chellappa,et al.  SFS based view synthesis for robust face recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[28]  P J. Phillips,et al.  Face Recognition Vendor Test 2000: Evaluation Report , 2001 .

[29]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[30]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  H Moon,et al.  Computational and Performance Aspects of PCA-Based Face-Recognition Algorithms , 2001, Perception.

[32]  Timothy F. Cootes,et al.  View-based active appearance models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[33]  Sami Romdhani,et al.  Face identification across different poses and illuminations with a 3D morphable model , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[34]  Ralph Gross,et al.  Eigen light-fields and face recognition across pose , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[35]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[36]  William H. Press,et al.  Numerical recipes in C , 2002 .

[37]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[38]  P. Jonathon Phillips,et al.  Face recognition vendor test 2002 , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[39]  Michael Jones,et al.  Multidimensional Morphable Models: A Framework for Representing and Matching Object Classes , 2004, International Journal of Computer Vision.

[40]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[41]  Thomas Vetter,et al.  Synthesis of Novel Views from a Single Face Image , 1998, International Journal of Computer Vision.