Feature fusion for facial landmark detection

Abstract Facial landmark detection is a crucial first step in facial analysis for biometrics and numerous other applications. However, it has proved to be a very challenging task due to the numerous sources of variation in 2D and 3D facial data. Although landmark detection based on descriptors of the 2D and 3D appearance of the face has been extensively studied, the fusion of such feature descriptors is a relatively under-studied issue. In this paper, a novel generalized framework for combining facial feature descriptors is presented, and several feature fusion schemes are proposed and evaluated. The proposed framework maps each feature into a similarity score and combines the individual similarity scores into a resultant score, used to select the optimal solution for a queried landmark. The evaluation of the proposed fusion schemes for facial landmark detection clearly indicates that a quadratic distance to similarity mapping in conjunction with a root mean square rule for similarity fusion achieves the best performance in accuracy, efficiency, robustness and monotonicity.

[1]  David Cristinacce,et al.  Automatic feature localisation with constrained local models , 2008, Pattern Recognit..

[2]  Sergios Theodoridis,et al.  Pattern Recognition , 1998, IEEE Trans. Neural Networks.

[3]  Andrew E. Johnson,et al.  Spin-Images: A Representation for 3-D Surface Matching , 1997 .

[4]  Arun Ross,et al.  Information fusion in biometrics , 2003, Pattern Recognit. Lett..

[5]  Berk Gökberk,et al.  Comparative Analysis of Decision-level Fusion Algorithms for 3D Face Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[6]  Alan C. Bovik,et al.  Passive Multimodal 2-D+3-D Face Recognition Using Gabor Features and Landmark Distances , 2011, IEEE Transactions on Information Forensics and Security.

[7]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[9]  K. Walker,et al.  View-based active appearance models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[10]  Ioannis A. Kakadiaris,et al.  3D Facial Landmark Detection under Large Yaw and Expression Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Anil K. Jain,et al.  Multimodal Facial Feature Extraction for Automatic 3D Face Recognition , 2005 .

[12]  Chris Boehnen,et al.  A Fast Multi-Modal Approach to Facial Feature Detection , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[13]  Ioannis A. Kakadiaris,et al.  Automatic 3D Facial Region Retrieval from Multi-pose Facial Datasets , 2009, 3DOR@Eurographics.

[14]  Sergios Theodoridis,et al.  Pattern Recognition, Third Edition , 2006 .

[15]  A. Ross,et al.  Level Fusion Using Hand and Face Biometrics , 2005 .

[16]  Tsz-Ho Yu,et al.  A Novel Genetic Algorithm for 3D Facial Landmark Localization , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[17]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[18]  Timothy F. Cootes,et al.  Statistical models of appearance for computer vision , 1999 .

[19]  Adel Guitouni,et al.  An Essay to Characterise Information Fusion Systems , 2006, 2006 9th International Conference on Information Fusion.

[20]  Anil K. Jain,et al.  Automatic feature extraction for multiview 3D face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[21]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Alice J. O'Toole,et al.  FRVT 2006 and ICE 2006 Large-Scale Experimental Results , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Arun Ross,et al.  Score normalization in multimodal biometric systems , 2005, Pattern Recognit..

[24]  Andrea J. van Doorn,et al.  Surface shape and curvature scales , 1992, Image Vis. Comput..

[25]  Ioannis A. Kakadiaris,et al.  Unified 3D face and ear recognition using wavelets on geometry images , 2008, Pattern Recognit..

[26]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[27]  Timothy F. Cootes,et al.  Statistical models of appearance for medical image analysis and computer vision , 2001, SPIE Medical Imaging.

[28]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[29]  Ioannis A. Kakadiaris,et al.  Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Arun Ross,et al.  Feature level fusion of hand and face biometrics , 2005, SPIE Defense + Commercial Sensing.

[31]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Anil K. Jain,et al.  Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Andrea Cavallaro,et al.  3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model , 2009, IEEE Transactions on Multimedia.

[34]  Chitra Dorai,et al.  COSMOS - A Representation Scheme for 3D Free-Form Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..