Biometric face recognition: from classical statistics to future challenges

Face recognition involves at least three major concepts from statistics: dimension reduction, feature extraction, and prediction. A selective review of algorithms, from seminal to state‐of‐the‐art, explores how these concepts persist as organizing principles in the field. Algorithms based directly upon classical statistical techniques include linear methods like principal component analysis and linear discriminant analysis. Nonlinear manifold methods, such as Laplacianfaces and Stiefel quotients, offer considerable performance improvements. Other noteworthy ideas include three‐dimensional morphable models, methods using local regions and/or alternative feature spaces (e.g., elastic bunch graph matching and local binary patterns) and sparse representation approaches. Opportunities for innovative statistical and collaborative research in face recognition are expanding in tandem with the growing complexity and diversity of applications. WIREs Comput Stat 2013, 5:288–308. doi: 10.1002/wics.1262

[1]  H. Hotelling The Generalization of Student’s Ratio , 1931 .

[2]  S. S. Wilks CERTAIN GENERALIZATIONS IN THE ANALYSIS OF VARIANCE , 1932 .

[3]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[4]  D. Lawley A GENERALIZATION OF FISHER'S z TEST , 1938 .

[5]  M. Bartlett A note on tests of significance in multivariate analysis , 1939, Mathematical Proceedings of the Cambridge Philosophical Society.

[6]  K. Pillai Some New Test Criteria in Multivariate Analysis , 1955 .

[7]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[8]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[9]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[10]  A. O'Toole,et al.  CONNECTIONIST APPROACHES TO VISUALLY-BASED FACIAL FEATURE EXTRACTION , 1988 .

[11]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[12]  Alex Pentland,et al.  A Bayesian similarity measure for direct image matching , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[13]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[14]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[15]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[16]  Alex Pentland,et al.  Probabilistic Visual Learning for Object Representation , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[19]  K. Etemad,et al.  Discriminant analysis for recognition of human face images , 1997 .

[20]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[21]  Ken-ichi Maeda,et al.  Face recognition using temporal image sequence , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[22]  Rama Chellappa,et al.  Discriminant analysis of principal components for face recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[23]  Kevin W. Bowyer,et al.  Empirical evaluation techniques in computer vision , 1998 .

[24]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[25]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[26]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[27]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[29]  T. Kanade,et al.  Combining Models and Exemplars for Face Recognition: An Illuminating Example , 2001 .

[30]  Alice J. O'Toole,et al.  Face recognition algorithms and the other-race effect: computational mechanisms for a developmental contact hypothesis , 2002, Cogn. Sci..

[31]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[32]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Sanghoon Kim,et al.  Computational Analysis of PCA-based Face Recognition Algorithms , 2003 .

[35]  Jian-Jun Zhang,et al.  Self quotient image for face recognition , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[36]  Zhi-Hua Zhou,et al.  Unified Locally Linear Embedding and Linear Discriminant Analysis Algorithm (ULLELDA) for Face Recognition , 2004, SINOBIOMETRICS.

[37]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[38]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[39]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Chengjun Liu,et al.  Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Wen Gao,et al.  Ensemble of Piecewise FDA Based on Spatial Histograms of Local (Gabor) Binary Patterns for Face Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[44]  Marios Savvides,et al.  Correlation Pattern Recognition for Face Recognition , 2006, Proceedings of the IEEE.

[45]  Patrick J. Flynn,et al.  Preliminary Face Recognition Grand Challenge Results , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[46]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[47]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Richard B. Darlington,et al.  Some New Test Criteria in Multivariate Analysis , 2007 .

[49]  Wen Gao,et al.  Histogram of Gabor Phase Patterns (HGPP): A Novel Object Representation Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[50]  B. K. Julsing,et al.  Face Recognition with Local Binary Patterns , 2012 .

[51]  Qiang Ji,et al.  A Comparative Study of Local Matching Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[52]  Xiaoyang Tan,et al.  Fusing Gabor and LBP Feature Sets for Kernel-Based Face Recognition , 2007, AMFG.

[53]  Wen Gao,et al.  Hierarchical Ensemble of Global and Local Classifiers for Face Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[54]  Alice J. O'Toole,et al.  Face Recognition Algorithms surpass humans matching faces across changes in illumination | NIST , 2007 .

[55]  Alice J. O'Toole,et al.  Face Recognition Algorithms Surpass Humans Matching Faces Over Changes in Illumination , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2016, Texts in Computer Science.

[57]  A. Hadid,et al.  The Local Binary Pattern Approach and its Applications to Face Analysis , 2008, 2008 First Workshops on Image Processing Theory, Tools and Applications.

[58]  J. Ross Beveridge,et al.  Grassmann Registration Manifolds for Face Recognition , 2008, ECCV.

[59]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[60]  Shu Liao,et al.  Face recognition with salient local gradient orientation binary patterns , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[61]  Matti Pietikäinen,et al.  (Multiscale) Local Phase Quantisation histogram discriminant analysis with score normalisation for robust face recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[62]  Jieping Ye,et al.  Finite Domain Constraint Solver Learning , 2009, IJCAI.

[63]  Bruce A. Draper,et al.  Principal Angles Separate Subject Illumination Spaces in YDB and CMU-PIE , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Massimo Tistarelli,et al.  Handbook of Remote Biometrics: for Surveillance and Security , 2009 .

[65]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Rama Chellappa,et al.  Handbook of Remote Biometrics , 2009, Advances in Pattern Recognition.

[67]  Yui Man Lui,et al.  Canonical Stiefel Quotient and its application to generic face recognition in illumination spaces , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[68]  Fernando De la Torre,et al.  Detecting depression from facial actions and vocal prosody , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[69]  Jie Chen,et al.  Fusing Local Patterns of Gabor Magnitude and Phase for Face Recognition , 2010, IEEE Transactions on Image Processing.

[70]  Chengjun Liu,et al.  Fusion of color, local spatial and global frequency information for face recognition , 2010, Pattern Recognit..

[71]  George W. Quinn,et al.  Report on the Evaluation of 2D Still-Image Face Recognition Algorithms , 2011 .

[72]  P. Jonathon Phillips,et al.  Improving Face Recognition Technology , 2011, Computer.

[73]  Julianne H. Ayyad,et al.  Recognizing people from dynamic and static faces and bodies: Dissecting identity with a fusion approach , 2010, Vision Research.

[74]  Chris H. Q. Ding,et al.  Linear Discriminant Analysis: New Formulations and Overfit Analysis , 2011, AAAI.

[75]  Bruce A. Draper,et al.  The Good, the Bad, and the Ugly Face Challenge Problem , 2012, Image and Vision Computing.

[76]  Peter N. Belhumeur,et al.  Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification , 2012, BMVC.

[77]  Bruce A. Draper,et al.  Preliminary studies on the Good, the Bad, and the Ugly face recognition challenge problem , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[78]  Alice J. O'Toole,et al.  Comparing face recognition algorithms to humans on challenging tasks , 2012, TAP.

[79]  Yong Man Ro,et al.  Local Color Vector Binary Patterns From Multichannel Face Images for Face Recognition , 2012, IEEE Transactions on Image Processing.

[80]  Vaidehi S. Natu,et al.  Unaware Person Recognition From the Body When Face Identification Fails , 2013, Psychological science.

[81]  Rolf P. Würtz,et al.  Elastic Bunch Graph Matching , 2014, Scholarpedia.