A Comparative Study of 3-D Face Recognition Under Expression Variations

Research in face recognition has continuously been challenged by extrinsic (head pose, lighting conditions) and intrinsic (facial expression, aging) sources of variability. While many survey papers on face recognition exist, in this paper, we focus on a comparative study of 3-D face recognition under expression variations. As a first contribution, 3-D face databases with expressions are listed, and the most important ones are briefly presented and their complexity is quantified using the iterative closest point (ICP) baseline recognition algorithm. This allows to rank the databases according to their inherent difficulty for face-recognition tasks. This analysis reveals that the FRGC v2 database can be considered as the most challenging because of its size, the presence of expressions and outliers, and the time lapse between the recordings. Therefore, we recommend to use this database as a reference database to evaluate (expression-invariant) 3-D face-recognition algorithms. We also determine and quantify the most important factors that influence the performance. It appears that performance decreases 1) with the degree of nonfrontal pose, 2) for certain expression types, 3) with the magnitude of the expressions, 4) with an increasing number of expressions, and 5) for a higher number of gallery subjects. Future 3-D face-recognition algorithms should be evaluated on the basis of all these factors. As the second contribution, a survey of published 3-D face-recognition methods that deal with expression variations is given. These methods are subdivided into three classes depending on the way the expressions are handled. Region-based methods use expression-stable regions only, while other methods model the expressions either using an isometric or a statistical model. Isometric models assume the deformation because of expression variation to be (locally) isometric, meaning that the deformation preserves lengths along the surface. Statistical models learn how the facial soft tissue deforms during expressions based on a training database with expression labels. Algorithmic performances are evaluated by the comparison of recognition rates for identification and verification. No statistical significant differences in class performance are found between any pair of classes.

[1]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Paul Suetens,et al.  Symmetric surface-feature based 3D face recognition for partial data , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[3]  Thomas Vetter,et al.  SHREC’08 entry: Shape based face recognition with a Morphable Model , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[4]  Witold Pedrycz,et al.  Face recognition: A study in information fusion using fuzzy integral , 2005, Pattern Recognit. Lett..

[5]  Daniel Rueckert,et al.  Evaluation of automatic 4D face recognition using surface and texture registration , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[6]  Maurício Pamplona Segundo,et al.  3D Face Recognition Using Simulated Annealing and the Surface Interpenetration Measure , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Yanfeng Sun,et al.  3D face recognition based on sparse representation , 2010, The Journal of Supercomputing.

[8]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[9]  Yiying Tong,et al.  Age-Invariant Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Marc Pollefeys,et al.  3D imaging for safety and security , 2007 .

[11]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Patrick J. Flynn,et al.  Preliminary Face Recognition Grand Challenge Results , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[13]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[14]  Alberto Del Bimbo,et al.  SHREC’08 entry: 3D face recognition using integral shape information , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[15]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[16]  Patrick J. Flynn,et al.  Assessment of Time Dependency in Face Recognition: An Initial Study , 2003, AVBPA.

[17]  Zhaohui Wu,et al.  3D Face Recognition using Mapped Depth Images , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[18]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[19]  MengChu Zhou,et al.  Image Ratio Features for Facial Expression Recognition Application , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Vinod Chandran,et al.  3D Face Recognition using Log-Gabor Templates , 2006, BMVC.

[21]  Willie Jones.  Plastic surgery 1, face recognition 0 , 2009, IEEE Spectrum.

[22]  Josef Kittler,et al.  3D Assisted Face Recognition: A Survey of 3D Imaging, Modelling and Recognition Approachest , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[23]  Frank B. ter Haar,et al.  A 3D face matching framework , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[24]  Alexander M. Bronstein,et al.  Three-Dimensional Face Recognition , 2005, International Journal of Computer Vision.

[25]  Maurício Pamplona Segundo,et al.  Noise versus Facial Expression on 3D Face Recognition , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[26]  H Moon,et al.  Computational and Performance Aspects of PCA-Based Face-Recognition Algorithms , 2001, Perception.

[27]  J. Jost.  Riemannian geometry and geometric analysis , 1995 .

[28]  Anil K. Jain,et al.  Deformation Modeling for Robust 3D Face Matching , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Vinay Bettadapura,et al.  Face Expression Recognition and Analysis: The State of the Art , 2012, ArXiv.

[30]  Xun Xu,et al.  Building Large Scale 3D Face Database for Face Analysis , 2007, MCAM.

[31]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[32]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Frank B. ter Haar,et al.  SHape REtrieval contest 2008: 3D face scans , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[34]  Jürgen Jost,et al.  Riemannian Geometry and Geometric Analysis, 5th Edition , 2008 .

[35]  Tieniu Tan,et al.  Automatic 3D face recognition from depth and intensity Gabor features , 2009, Pattern Recognit..

[36]  Cristina Conde,et al.  Multimodal 2D, 2.5D & 3D Face Verification , 2006, 2006 International Conference on Image Processing.

[37]  Patrick J. Flynn,et al.  3D Face Recognition with Region Committee Voting , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[38]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[39]  Chris Boehnen,et al.  3D Face Recognition Using 3D Alignment for PCA , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[40]  Paul Suetens,et al.  Isometric deformation modeling using singular value decomposition for 3D expression-invariant face recognition , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[41]  Guodong Guo,et al.  Learning from examples in the small sample case: face expression recognition , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Li Li,et al.  3D face recognition by constructing deformation invariant image , 2008, Pattern Recognit. Lett..

[43]  Andrea Cavallaro,et al.  SHREC’08 entry: Registration and retrieval of 3D faces using a Point Distribution Model , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[44]  Moulay A. Akhloufi,et al.  Infrared Face Recognition Using Distance Transforms , 2008 .

[45]  Mohammed Bennamoun,et al.  A survey of approaches to three-dimensional face recognition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[46]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Faisal R. Al-Osaimi,et al.  An Expression Deformation Approach to Non-rigid 3D Face Recognition , 2009, International Journal of Computer Vision.

[48]  Tieniu Tan,et al.  Learning Effective Intrinsic Features to Boost 3D-Based Face Recognition , 2006, ECCV.

[49]  Alexander M. Bronstein,et al.  Expression-Invariant Representations of Faces , 2007, IEEE Transactions on Image Processing.

[50]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[51]  Sunghee Choi,et al.  The power crust , 2001, SMA '01.

[52]  Patrick J. Flynn,et al.  Face Recognition Using 2D and 3D Facial Data , 2003 .

[53]  S. Malassiotis,et al.  Expression-Compensated 3D Face Recognition with Geodesically Aligned Bilinear Models , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[54]  A. Bovik,et al.  Advances and Challenges in 3 D and 2 D + 3 D Human Face Recognition , 2007 .

[55]  Rama Chellappa,et al.  Face Processing: Advanced Modeling and Methods , 2006, J. Electronic Imaging.

[56]  Thomas David Heseltine,et al.  Face recognition : two-dimensional and three-dimensional techniques , 2005 .

[57]  L. Farkas.  Anthropometry of the head and face , 1994 .

[58]  Dirk Vandermeulen,et al.  Objective 3D face recognition: Evolution, approaches and challenges. , 2010, Forensic science international.

[59]  Zhaohui Wu,et al.  3D Face Recognition in the Presence of Expression: A Guidance-based Constraint Deformation Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Michael G. Strintzis,et al.  3-D Face Recognition With the Geodesic Polar Representation , 2007, IEEE Transactions on Information Forensics and Security.

[61]  Yunhong Wang,et al.  3D Face Recognition Based on Local Shape Patterns and Sparse Representation Classifier , 2011, MMM.

[62]  Hao Zhang,et al.  Expression-insensitive 3D face recognition using sparse representation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  P. Jonathon Phillips,et al.  Face recognition vendor test 2002 , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[64]  B. H. Brinkmann,et al.  A book review of G.R.J. Swennen, F Schutyser, J-E Hausamen: “Three-dimensional cephalometry—a color atlas and manual” , 2006, European Journal of Plastic Surgery.

[65]  Sridha Sridharan,et al.  Combined 2D/3D Face Recognition Using Log-Gabor Templates , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[66]  Patrick J. Flynn,et al.  A Region Ensemble for 3-D Face Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[67]  Alberto Del Bimbo,et al.  Description and retrieval of 3D face models using iso-geodesic stripes , 2006, MIR '06.

[68]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[69]  Mohammed Bennamoun,et al.  Region-based Matching for Robust 3D Face Recognition , 2005, BMVC.

[70]  Filip Schutyser,et al.  Three-dimensional cephalometry : a color atlas and manual , 2006 .

[71]  Phalguni Gupta,et al.  An Efficient 3D Face Recognition Algorithm , 2009, 2009 3rd International Conference on New Technologies, Mobility and Security.

[72]  Alexander M. Bronstein,et al.  Expression-invariant face recognition via spherical embedding , 2005, IEEE International Conference on Image Processing 2005.

[73]  Frank B. ter Haar,et al.  SHREC’08 entry: 3D face recognition using facial contour curves , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[74]  Alexander M. Bronstein,et al.  Robust Expression-Invariant Face Recognition from Partially Missing Data , 2006, ECCV.

[75]  Dirk Vandermeulen,et al.  Surface representations for 3D face recognition , 2010 .

[76]  Patrick J. Flynn,et al.  Using multi-instance enrollment to improve performance of 3D face recognition , 2008, Comput. Vis. Image Underst..

[77]  Tieniu Tan,et al.  Automatic 3D Face Recognition Using Discriminant Common Vectors , 2006, ICB.

[78]  Jim Austin,et al.  Three-dimensional face recognition using combinations of surface feature map subspace components , 2008, Image Vis. Comput..

[79]  Paul Suetens,et al.  Meshfree mathematical surface representations and their use in biometrics , 2009 .

[80]  Faisal R. Al-Osaimi,et al.  Expression-invariant Non-rigid 3D Face Recognition: A Robust Approach to Expression-aware Morphing , 2008 .

[81]  Hao Zhang,et al.  Adapting Geometric Attributes for Expression-Invariant 3D Face Recognition , 2007, IEEE International Conference on Shape Modeling and Applications 2007 (SMI '07).

[82]  J. Kittler,et al.  Algorithms for 3D-Assisted Face Recognition , 2006, 2006 IEEE 14th Signal Processing and Communications Applications.

[83]  Peter Claes,et al.  How Different is Different? Criterion and Sensitivity in Face-Space , 2011, Front. Psychology.

[84]  Berk Gökberk,et al.  3D face recognition for biometric applications , 2005, 2005 13th European Signal Processing Conference.

[85]  Zhaohui Wu,et al.  Exploring Facial Expression Effects in 3D Face Recognition Using Partial ICP , 2006, ACCV.

[86]  Alexander M. Bronstein,et al.  Expression-Invariant 3D Face Recognition , 2003, AVBPA.

[87]  A. Bovik,et al.  Three Dimensional Face Recognition Using Iso-Geodesic and Iso-Depth Curves , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[88]  Paul Suetens,et al.  Fusion of an Isometric Deformation Modeling Approach Using Spectral Decomposition and a Region-Based Approach Using ICP for Expression-Invariant 3D Face Recognition , 2010, 2010 20th International Conference on Pattern Recognition.

[89]  P. Suetens,et al.  3D Face Recognition using Point Cloud Kernel Correlation , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[90]  Feng Han,et al.  3D human face recognition using point signature , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[91]  Evangelos E. Milios,et al.  Matching range images of human faces , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[92]  Jim Austin,et al.  Three-Dimensional Face Recognition Using Surface Space Combinations , 2004, BMVC.

[93]  Patrick J. Flynn,et al.  Multiple Nose Region Matching for 3D Face Recognition under Varying Facial Expression , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[94]  L. Akarun,et al.  A 3D Face Recognition System for Expression and Occlusion Invariance , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[95]  Mohammed Bennamoun,et al.  2D and 3D Multimodal Hybrid Face Recognition , 2006, ECCV.

[96]  Ron Kimmel,et al.  Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[97]  Wei-Yang Lin,et al.  Fusion of Multiple Facial Regions for Expression-Invariant Face Recognition , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[98]  J A Sethian,et al.  Computing geodesic paths on manifolds. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[99]  K.W. Bowyer,et al.  Using a Multi-Instance Enrollment Representation to Improve 3D Face Recognition , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[100]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[101]  Frank B. ter Haar,et al.  A 3D face matching framework for facial curves , 2009, Graph. Model..

[102]  Gordon Erlebacher,et al.  A novel technique for face recognition using range imaging , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[103]  Faouzi Ghorbel,et al.  3D Face Recognition using ICP and Geodesic Computation Coupled Approach , 2008 .

[104]  Jake K. Aggarwal,et al.  Three dimensional face recognition based on geodesic and Euclidean distances , 2007, Electronic Imaging.

[105]  N. Poh,et al.  Multibiometrics for Identity Authentication: Issues, Benefits and Challenges , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[106]  Patrick J. Flynn,et al.  A survey of approaches to three-dimensional face recognition , 2004, ICPR 2004.

[107]  Patrick J. Flynn,et al.  Adaptive Rigid Multi-region Selection for Handling Expression Variation in 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[108]  Andrew Pressley.  The first fundamental form , 2010 .

[109]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[110]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[111]  Dong Xu,et al.  SHREC’08 entry: 3D face recognition using moment invariants , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[112]  Anuj Srivastava,et al.  Three-Dimensional Face Recognition Using Shapes of Facial Curves , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[113]  Daniel Riccio.  State of the art in 3d face recognition , 2005 .

[114]  Tieniu Tan,et al.  Robust 3D Face Recognition Using Learned Visual Codebook , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[115]  Jake K. Aggarwal,et al.  3D Face Recognition Founded on the Structural Diversity of Human Faces , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[116]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[117]  P. Ekman.  Universals and cultural differences in facial expressions of emotion. , 1972 .

[118]  Remco C. Veltkamp,et al.  A Survey of 3D Face Recognition Methods , 2005, AVBPA.

[119]  Hamid Krim,et al.  3D face recognition based on evolution of iso-geodesic distance curves , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[120]  Alan C. Bovik,et al.  Texas 3D Face Recognition Database , 2010, 2010 IEEE Southwest Symposium on Image Analysis & Interpretation (SSIAI).

[121]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[122]  Ioannis A. Kakadiaris,et al.  Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[123]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[124]  H. Krim,et al.  3D Face Recognition using Euclidean Integral Invariants Signature , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.