Landmark Localisation in 3D Face Data

A comparison of several approaches that use graph matching and cascade filtering for landmark localization in 3D face data is presented. For the first method, we apply the structural graph matching algorithm “relaxation by elimination” using a simple “distance to local plane” node property and a “Euclidean distance” arc property. After the graph matching process has eliminated unlikely candidates, the most likely triplet is selected, by exhaustive search, as the minimum Mahalanobis distance over a six dimensional space, corresponding to three node variables and three arc variables. A second method uses state-of-the-art pose-invariant feature descriptors embedded into a cascade filter to localize the nose tip. After that, local graph matching is applied to localize the inner eye corners. We evaluate our systems by computing root mean square errors of estimated landmark locations against ground truth landmark localizations within the 3D Face Recognition Grand Challenge database. Our best system, which uses a novel pose-invariant shape descriptor, scores 99.77% successful localization of the nose and 96.82% successful localization of the eyes.

[1]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[2]  William J. Christmas,et al.  Structural Matching in Computer Vision Using Probabilistic Relaxation , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[4]  Joseph A. Gallian,et al.  A Dynamic Survey of Graph Labeling , 2009, The Electronic Journal of Combinatorics.

[5]  Maurício Pamplona Segundo,et al.  Automatic 3D facial segmentation and landmark detection , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[6]  James F. O'Brien,et al.  Shape transformation using variational implicit functions , 1999, SIGGRAPH 1999.

[7]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Albert Ali Salah,et al.  3D Facial Feature Localization for Registration , 2006, MRCS.

[9]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Xuelong Li,et al.  General Tensor Discriminant Analysis and Gabor Features for Gait Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Azriel Rosenfeld,et al.  Scene Labeling by Relaxation Operations , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Cristina Conde,et al.  Automatic 3D Face Feature Points Extraction with Spin Images , 2006, ICIAR.

[14]  Tinashe Mutsvangwa,et al.  Morphometric analysis of facial landmark data to characterize the facial phenotype associated with fetal alcohol syndrome , 2007, Journal of anatomy.

[15]  A. Mehrabian Communication without words , 1968 .

[16]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Yali Amit,et al.  Joint Induction of Shape Features and Tree Classifiers , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Carlos D. Castillo,et al.  Using Stereo Matching with General Epipolar Geometry for 2D Face Recognition across Pose , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  A. Yuille,et al.  Two- and Three-Dimensional Patterns of the Face , 2001 .

[20]  Jim Austin,et al.  Graph matching by neural relaxation , 2005, Neural Computing & Applications.

[21]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[22]  Takeo Kanade,et al.  Computer recognition of human faces , 1980 .

[23]  Patrick J. Flynn,et al.  Face Recognition Using 2D and 3D Facial Data , 2003 .

[24]  Anil K. Jain,et al.  Detection of Anchor Points for 3D Face Veri.cation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[25]  Chitra Dorai,et al.  COSMOS - A Representation Scheme for 3D Free-Form Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Leslie Greengard,et al.  A fast algorithm for particle simulations , 1987 .

[27]  Keith E. Price Hierarchical matching using relaxation , 1986, Comput. Vis. Graph. Image Process..

[28]  Veikko Surakka,et al.  Feature-based detection of facial landmarks from neutral and expressive facial images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Rama Chellappa,et al.  Unconstrained Face Recognition (International Series on Biometrics) , 2005 .

[30]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Shaohua Kevin Zhou,et al.  A Binary Decision Tree Implementation of a Boosted Strong Classifier , 2005, AMFG.

[32]  Horst Bunke,et al.  Face recognition using range images , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[33]  Remco C. Veltkamp,et al.  A Survey of 3D Face Recognition Methods , 2005, AVBPA.

[34]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[35]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[36]  Chin Seng Chua,et al.  Point Signatures: A New Representation for 3D Object Recognition , 1997, International Journal of Computer Vision.

[37]  C. Darwin The Expression of the Emotions in Man and Animals , .

[38]  Gaile G. Gordon,et al.  Face recognition based on depth and curvature features , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Patrick J. Flynn,et al.  A Region Ensemble for 3-D Face Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[40]  Shaohua Zhou,et al.  Unconstrained Face Recognition , 2005 .

[41]  Martin A. Fischler,et al.  The Representation and Matching of Pictorial Structures , 1973, IEEE Transactions on Computers.

[42]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[43]  Rama Chellappa,et al.  Face Processing: Advanced Modeling and Methods , 2006, J. Electronic Imaging.

[44]  Josef Kittler,et al.  Combining Evidence in Probabilistic Relaxation , 1989, Int. J. Pattern Recognit. Artif. Intell..

[45]  Shaogang Gong,et al.  Analysis and Modelling of Faces and Gestures , 2008 .

[46]  Fanglin Chen,et al.  A Novel Algorithm for Detecting Singular Points from Fingerprint Images , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Patrick J. Flynn,et al.  Multiple Nose Region Matching for 3D Face Recognition under Varying Facial Expression , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  M. Romero-Huertas,et al.  3D Facial Landmark Localisation by Matching Simple Descriptors , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[49]  Alberto Del Bimbo,et al.  Spin images for retrieval of 3D objects by local and global similarity , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[50]  Reyes Enciso,et al.  Precision, repeatability and validation of indirect 3D anthropometric measurements with light-based imaging techniques , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[51]  John Daugman,et al.  Statistical Richness of Visual Phase Information: Update on Recognizing Persons by Iris Patterns , 2001, International Journal of Computer Vision.

[52]  L. Farkas Anthropometry of the head and face , 1994 .

[53]  Richard K. Beatson,et al.  Surface interpolation with radial basis functions for medical imaging , 1997, IEEE Transactions on Medical Imaging.

[54]  Nick Pears,et al.  Point-pair descriptors for 3D facial landmark localisation , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[55]  L. Akarun,et al.  A 3D Face Recognition System for Expression and Occlusion Invariance , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[56]  Michael G. Strintzis,et al.  Integration of 2D and 3D images for enhanced face authentication , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[57]  Ramesh C. Jain,et al.  Three-dimensional object recognition , 1985, CSUR.

[58]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[59]  Alice J. O'Toole,et al.  FRVT 2006 and ICE 2006 Large-Scale Experimental Results , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[61]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[62]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  R. Franke Scattered data interpolation: tests of some methods , 1982 .

[64]  Jake K. Aggarwal,et al.  3D Face Recognition Founded on the Structural Diversity of Human Faces , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Olivier D. Faugeras,et al.  Semantic Description of Aerial Images Using Stochastic Labeling , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Ping Yan,et al.  Biometric Recognition Using 3D Ear Shape , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Anil K. Jain,et al.  Automatic feature extraction for multiview 3D face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[68]  Veikko Surakka,et al.  Automatic Detection of Facial Landmarks from AU-coded Expressive Facial Images , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[69]  Afzal Godil,et al.  Facial Shape Analysis and Sizing System , 2009, HCI.

[70]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[71]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[72]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Stefano Soatto,et al.  Hybrid Dynamical Models of Human Motion for the Recognition of Human Gaits , 2009, International Journal of Computer Vision.

[75]  Chen Chen,et al.  Face Personalization: Animated Face Modeling Approach using Radial Basis Function , 2005, TENCON 2005 - 2005 IEEE Region 10 Conference.

[76]  Alexander M. Bronstein,et al.  Three-Dimensional Face Recognition , 2005, International Journal of Computer Vision.

[77]  I. Masuda,et al.  3D facial image analysis for human identification , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[78]  K.W. Bowyer,et al.  The Best Bits in an Iris Code , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Chin-Seng Chua,et al.  Facial feature detection and face recognition from 2D and 3D images , 2002, Pattern Recognit. Lett..

[80]  Nick Pears,et al.  From 3D Point Clouds to Pose-Normalised Depth Maps , 2010, International Journal of Computer Vision.

[81]  Li Bai,et al.  Line feature detection from 3D point clouds via adaptive CS-RBFs shape reconstruction and multistep vertex normal manipulation , 2005, International Conference on Computer Graphics, Imaging and Visualization (CGIV'05).

[82]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Data Mining Researchers , 2003 .

[83]  Tieniu Tan,et al.  Combining local features for robust nose location in 3D facial data , 2006, Pattern Recognit. Lett..

[84]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[85]  Edwin R. Hancock,et al.  Structural Matching by Discrete Relaxation , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[86]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[87]  Gérard G. Medioni,et al.  Face modeling and recognition in 3-D , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[88]  Nick Pears RBF shape histograms and their application to 3D face processing , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[89]  Aleix M. Martínez,et al.  Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  Hui Chen,et al.  Human Ear Recognition in 3D , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[91]  Rafael C. González,et al.  Digital image processing using MATLAB , 2006 .

[92]  Thomas S. Huang,et al.  Image processing , 1971 .

[93]  Mohammed Bennamoun,et al.  Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition , 2007, International Journal of Computer Vision.

[94]  Matthew Stone,et al.  An anthropometric face model using variational techniques , 1998, SIGGRAPH.

[95]  H. Quynh Dinh,et al.  Multi-Resolution Spin-Images , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[96]  Maurício Pamplona Segundo,et al.  3D Face Recognition Using Simulated Annealing and the Surface Interpenetration Measure , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.