Automatic Interpretation and Coding of Face Images Using Flexible Models

Face images are difficult to interpret because they are highly variable. Sources of variability include individual appearance, 3D pose, facial expression, and lighting. We describe a compact parametrized model of facial appearance which takes into account all these sources of variability. The model represents both shape and gray-level appearance, and is created by performing a statistical analysis over a training set of face images. A robust multiresolution search algorithm is used to fit the model to faces in new images. This allows the main facial features to be located, and a set of shape, and gray-level appearance parameters to be recovered. A good approximation to a given face can be reconstructed using less than 100 of these parameters. This representation can be used for tasks such as image coding, person identification, 3D pose recovery, gender recognition, and expression recognition. Experimental results are presented for a database of 690 face images obtained under widely varying conditions of 3D pose, lighting, and facial expression. The system performs well on all the tasks listed above.

[1]  D. F. Morrison,et al.  Multivariate Statistical Methods , 1968 .

[2]  P. Ekman,et al.  Unmasking the Face: A Guide to Recognizing Emotions From Facial Expressions , 1975 .

[3]  H. Sackeim,et al.  Emotions are expressed more intensely on the left side of the face. , 1978, Science.

[4]  B. Manly.  Multivariate Statistical Methods : A Primer , 1986 .

[5]  Alan L. Yuille,et al.  Feature extraction from faces using deformable templates , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  M. K. Fleming,et al.  Categorization of faces using unsupervised feature extraction , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[8]  Garrison W. Cottrell,et al.  EMPATH: Face, Emotion, and Gender Recognition Using Holons , 1990, NIPS.

[9]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  J. B. Waite,et al.  An application of active contour models to head boundary location , 1990, BMVC.

[11]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[12]  Ian Craw,et al.  Face Recognition by Computer , 1992, BMVC.

[13]  Ian Craw,et al.  Finding Face Features , 1992, ECCV.

[14]  Ian Craw,et al.  Blink Rate Monitoring for a Driver Awareness System , 1992, BMVC.

[15]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[17]  Timothy F. Cootes,et al.  Building and using flexible models incorporating grey-level information , 1993, 1993 (4th) International Conference on Computer Vision.

[18]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Timothy F. Cootes,et al.  The Use of Active Shape Models for Locating Structures in Medical Images , 1993, IPMI.

[20]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Timothy F. Cootes,et al.  Automatic tracking, coding and reconstruction of human faces, using flexible appearance models , 1994 .

[22]  Timothy F. Cootes,et al.  Multi-resolution search with active shape models , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[23]  Timothy F. Cootes,et al.  Use of active shape models for locating structures in medical images , 1994, Image Vis. Comput..

[24]  Chil-Woo Lee,et al.  Detection and pose estimation of human face with synthesized image models , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[25]  Roberto Cipolla,et al.  Estimating gaze from a single view of a face , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[26]  Larry Davis,et al.  Recognizing facial expressions by spatio-temporal analysis , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[27]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Münevver Köküer,et al.  A model-based codec with potential for deaf communication , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 2 - Conference B: Computer Vision & Image Processing. (Cat. No.94CH3440-5).

[29]  Timothy F. Cootes,et al.  Automatic face identification system using flexible appearance models , 1995, Image Vis. Comput..

[30]  T. F. CootesWolfson.  Classifying Variable Objects Using A Flexible Shape Model , 1995 .

[31]  Timothy F. Cootes,et al.  A unified approach to coding and interpreting face images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[32]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[33]  I Biederman,et al.  To what extent can matching algorithms based on direct outputs of spatial filters account for human object recognition? , 1996, Spatial vision.

[34]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..