Pose and Expression Independent Facial Landmark Localization Using Dense-SURF and the Hausdorff Distance

We present an approach to automatic localization of facial feature points which deals with pose, expression, and identity variations combining 3D shape models with local image patch classification. The latter is performed by means of densely extracted SURF-like features, which we call DU-SURF, while the former is based on a multiclass version of the Hausdorff distance to address local classification errors and nonvisible points. The final system is able to localize facial points in real-world scenarios, dealing with out of plane head rotations, expression changes, and different lighting conditions. Extensive experimentation with the proposed method has been carried out showing the superiority of our approach with respect to other state-of-the-art systems. Finally, DU-SURF features have been compared with other modern features and we experimentally demonstrate their competitive classification accuracy and computational efficiency.

[1]  Ioannis Pitas,et al.  Facial feature detection using distance vector fields , 2009, Pattern Recognit..

[2]  Lionel Prevost,et al.  Real-time facial feature localization by combining space displacement neural networks , 2008, Pattern Recognit. Lett..

[3]  Qiang Ji,et al.  Multiview Facial Feature Tracking with a Multi-modal Probabilistic Model , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[4]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[7]  Takeo Kanade,et al.  A Generative Shape Regularization Model for Robust Face Alignment , 2008, ECCV.

[8]  J. Crowley,et al.  Estimating Face orientation from Robust Detection of Salient Facial Structures , 2004 .

[9]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Trevor Darrell,et al.  On modelling nonlinear shape-and-texture appearance manifolds , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[12]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[13]  Timothy F. Cootes,et al.  Locating Facial Features and Pose Estimation Using a 3D Shape Model , 2009, ISVC.

[14]  Daniel P. Huttenlocher,et al.  Distance Transforms of Sampled Functions , 2012, Theory Comput..

[15]  Bülent Sankur,et al.  Robust facial landmarking for registration , 2007, Ann. des Télécommunications.

[16]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[17]  Maja Pantic,et al.  Fully automatic facial feature point detection using Gabor feature based boosted classifiers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[18]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[20]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[21]  Diego A. Socolinsky,et al.  Appearance-Based Facial Recognition Using Visible and Thermal Imagery: A Comparative Study , 2006 .

[22]  Turgay Çelik,et al.  Facial feature extraction using complex dual-tree wavelet transform , 2008, Comput. Vis. Image Underst..

[23]  James L. Crowley,et al.  Facial features detection robust to pose, illumination and identity , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[24]  Volker Blanz,et al.  Component-Based Face Recognition with 3D Morphable Models , 2003, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[25]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[26]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[27]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Sami Romdhani,et al.  3D Probabilistic Feature Point Model for Object Detection and Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Maja Pantic,et al.  Facial point detection using boosted regression and graph models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Jing Xiao,et al.  Real-time combined 2D+3D active appearance models , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[31]  Timothy F. Cootes,et al.  A Multi-Stage Approach to Facial Feature Detection , 2004, BMVC.

[32]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[34]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[35]  David Cristinacce,et al.  Automatic feature localisation with constrained local models , 2008, Pattern Recognit..

[36]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Takeo Kanade,et al.  3D Alignment of Face in a Single Image , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[38]  Osamu Yamaguchi,et al.  Facial feature localization using weighted vector concentration approach , 2010, Image Vis. Comput..

[39]  Lei Zhang,et al.  3D shape constraint for facial feature localization using probabilistic-like output , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[40]  David J. Kriegman,et al.  Localizing parts of faces using a consensus of exemplars , 2011, CVPR.

[41]  Sethuraman Panchanathan,et al.  Framework for performance evaluation of face recognition algorithms , 2002, SPIE ITCom.

[42]  Pengfei Shi,et al.  Facial Components Detection with Boosting and Geometric Constraints , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[43]  David J. Kriegman,et al.  Localizing Parts of Faces Using a Consensus of Exemplars , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[45]  Veikko Surakka,et al.  Feature-based detection of facial landmarks from neutral and expressive facial images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[47]  Luigi Cinque,et al.  Improved Statistical Techniques for Multi-part Face Detection and Recognition , 2009, SCIA.

[48]  Shaogang Gong,et al.  A Multi-View Nonlinear Active Shape Model Using Kernel PCA , 1999, BMVC.

[49]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  Klaus J. Kirchberg,et al.  Robust Face Detection Using the Hausdorff Distance , 2001, AVBPA.

[51]  Thomas Vetter,et al.  Estimating Coloured 3D Face Models from Single Images: An Example Based Approach , 1998, ECCV.

[52]  Aleix M. Martínez,et al.  Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[54]  Timothy F. Cootes,et al.  View-based active appearance models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[55]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[56]  Yang Wang,et al.  Robust facial feature tracking under varying face pose and facial expression , 2007, Pattern Recognit..

[57]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[58]  Vijayan K. Asari,et al.  Pose Angle Determination by Face, Eyes and Nose Localization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[59]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[60]  Cristina Conde,et al.  Multimodal 2D, 2.5D & 3D Face Verification , 2006, 2006 International Conference on Image Processing.

[61]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[62]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[63]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.