A Statistical Method for 2-D Facial Landmarking

Many facial-analysis approaches rely on robust and accurate automatic facial landmarking to correctly function. In this paper, we describe a statistical method for automatic facial-landmark localization. Our landmarking relies on a parsimonious mixture model of Gabor wavelet features, computed in coarse-to-fine fashion and complemented with a shape prior. We assess the accuracy and the robustness of the proposed approach in extensive cross-database conditions conducted on four face data sets (Face Recognition Grand Challenge, Cohn-Kanade, Bosphorus, and BioID). Our method has 99.33% accuracy on the Bosphorus database and 97.62% accuracy on the BioID database on the average, which improves the state of the art. We show that the method is not significantly affected by low-resolution images, small rotations, facial expressions, and natural occlusions such as beard and mustache. We further test the goodness of the landmarks in a facial expression recognition application and report landmarking-induced improvement over baseline on two separate databases for video-based expression recognition (Cohn-Kanade and BU-4DFE).

[1]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[4]  Baochang Zhang,et al.  Gabor feature constrained statistical model for efficient landmark localization and face recognition , 2009, Pattern Recognit. Lett..

[5]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[6]  Bülent Sankur,et al.  Robust facial landmarking for registration , 2007, Ann. des Télécommunications.

[7]  Thomas S. Huang,et al.  Connected vibrations: a modal analysis approach for non-rigid motion tracking , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[8]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[9]  Albert Ali Salah,et al.  Incremental mixtures of factor analysers , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[10]  Maja Pantic,et al.  Facial point detection using boosted regression and graph models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Maja Pantic,et al.  Fully automatic facial feature point detection using Gabor feature based boosted classifiers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[12]  C. Goodall.  Procrustes methods in the statistical analysis of shape , 1991 .

[13]  Peter H. Tu,et al.  Automatic facial landmark labeling with minimal supervision , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Timothy F. Cootes,et al.  Feature Detection and Tracking with Constrained Local Models , 2006, BMVC.

[15]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[18]  Jiri Matas,et al.  Feature-based affine-invariant localization of faces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Anil K. Jain,et al.  Detection of Anchor Points for 3D Face Veri.cation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[20]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  M. Romero-Huertas,et al.  3D Facial Landmark Localisation by Matching Simple Descriptors , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[22]  N. Sebe,et al.  Facial Expression Recognition: A Fully Integrated Approach , 2007, 14th International Conference of Image Analysis and Processing - Workshops (ICIAPW 2007).

[23]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Takeo Kanade,et al.  A Generative Shape Regularization Model for Robust Face Alignment , 2008, ECCV.

[25]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[26]  Xiaoming Liu,et al.  Video-based face model fitting using Adaptive Active Appearance Model , 2010, Image Vis. Comput..

[27]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Jian Liu,et al.  Active Appearance Models Fitting with Occlusion , 2007, EMMCVPR.

[29]  Saman K. Halgamuge,et al.  Optimised landmark model matching for face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[30]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[31]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[32]  Fred Nicolls,et al.  Locating Facial Features with an Extended Active Shape Model , 2008, ECCV.

[33]  Osamu Yamaguchi,et al.  Facial feature localization using weighted vector concentration approach , 2010, Image Vis. Comput..

[34]  Lei Zhang,et al.  3D shape constraint for facial feature localization using probabilistic-like output , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[35]  Timothy F. Cootes,et al.  Statistical models of appearance for computer vision , 1999 .

[36]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[37]  A. A. Salah,et al.  Registration of 3D Face Scans with Average Face Models , 2008 .

[38]  Paola Campadelli,et al.  A face recognition system based on automatically determined facial fiducial points , 2006, Pattern Recognit..

[39]  Emmanuel Dellandréa,et al.  A 3D Statistical Facial Feature Model and Its Application on Locating Facial Landmarks , 2009, ACIVS.

[40]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).