Local Representation of Facial Features

Feature extraction is one of the fundamental tasks in computer vision and image processing. Respectively, the task of selecting the best set of features to describe faces for recognition, verification, localization, or detection, is a fundamental problem in face biometrics. In this chapter, we review the most popular and successful features for face biometrics. In general, one should include complete algorithms when comparing the features, but certain extraction methods seem to maintain popularity due to their continuous success in various methods and approaches in biometrics and other fields of computer vision and image processing. This chapter specifically describes in more details two prominent local facial features, the first one based on Gabor filter responses, and the second on more recently proposed local binary patterns (LBPs).

[1]  Chi-Ho Chan.  Multi-scale local Binary Pattern Histogram for Face Recognition , 2007, ICB.

[2]  Matti Pietikäinen,et al.  Boosted multi-resolution spatiotemporal descriptors for facial expression recognition , 2009, Pattern Recognit. Lett..

[3]  Wen Gao,et al.  Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Matti Pietikäinen,et al.  Face Analysis Using Local Binary Patterns , 2008 .

[5]  Shengcai Liao,et al.  Learning Multi-scale Block Local Binary Patterns for Face Recognition , 2007, ICB.

[6]  Wen Gao,et al.  Hierarchical Ensemble of Global and Local Classifiers for Face Recognition , 2009, IEEE Trans. Image Process..

[7]  Mohammed Bennamoun,et al.  Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition , 2007, International Journal of Computer Vision.

[8]  Rainer Stiefelhagen,et al.  Analysis of Local Appearance-Based Face Recognition: Effects of Feature Selection and Feature Normalization , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[9]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Takeo Kanade,et al.  Picture Processing System by Computer Complex and Recognition of Human Faces , 1974 .

[11]  Qiang Ji,et al.  Multi-view face and eye detection using discriminant features , 2007, Comput. Vis. Image Underst..

[12]  Matti Pietikäinen,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON MULTIMEDIA 1 Lipreading with Local Spatiotemporal Descriptors , 2022 .

[13]  LinLin Shen,et al.  Face authentication test on the BANCA database , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[14]  Shengcai Liao,et al.  Using Near-Infrared Images , 2007 .

[15]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, CAIP.

[16]  Nicolas Pinto,et al.  How far can you get with a modern face recognition test set using only simple features? , 2009, CVPR.

[17]  Yi-Ping Hung,et al.  Face verification and identification using Facial Trait Code , 2009, CVPR.

[18]  Liya Ding,et al.  Precise detailed detection of faces and facial features , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Joni-Kristian Kämäräinen,et al.  Simple Gabor feature space for invariant object recognition , 2004, Pattern Recognit. Lett..

[21]  Gang Hua,et al.  A robust elastic and partial matching metric for face recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[22]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Yi-Ping Hung,et al.  Face verification and identification using Facial Trait Code , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[25]  John Daugman,et al.  Neural networks for image transformation, analysis, and compression , 1988, Neural Networks.

[26]  M. Pietikäinen,et al.  Facial Expression Recognition with Local Binary Patterns and Linear Programming 1 , 2005 .

[27]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[28]  Baochang Zhang,et al.  Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor , 2010, IEEE Transactions on Image Processing.

[29]  Matthew A. Brown,et al.  Learning Local Image Descriptors , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Ingrid Daubechies,et al.  The wavelet transform, time-frequency localization and signal analysis , 1990, IEEE Trans. Inf. Theory.

[32]  Josef Kittler,et al.  Object Localisation Using Generative Probability Model for Spatial Constellation and Local Image Features , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[33]  Xiaoyang Tan,et al.  Fusing Gabor and LBP Feature Sets for Kernel-Based Face Recognition , 2007, AMFG.

[34]  Jiri Matas,et al.  Feature-based affine-invariant localization of faces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[36]  Joni-Kristian Kämäräinen,et al.  Invariance properties of Gabor filter-based features-overview and applications , 2006, IEEE Transactions on Image Processing.

[37]  Anil K. Jain,et al.  Pores and Ridges: Fingerprint Matching Using Level 3 Features , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[38]  Wen Gao,et al.  Ensemble of Piecewise FDA Based on Spatial Histograms of Local (Gabor) Binary Patterns for Face Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[39]  M. Pietikäinen,et al.  FACIAL EXPRESSION RECOGNITION WITH LOCAL BINARY PATTERNS AND LINEAR PROGRAMMING , 2004 .

[40]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[41]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[42]  Paola Campadelli,et al.  A face recognition system based on automatically determined facial fiducial points , 2006, Pattern Recognit..

[43]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[44]  Oscar Déniz-Suárez,et al.  A comparison of face and facial feature detectors based on the Viola–Jones general object detection framework , 2011, Machine Vision and Applications.

[45]  Matti Pietikäinen,et al.  Pixelwise Local Binary Pattern Models of Faces Using Kernel Density Estimation , 2009, ICB.

[46]  Tieniu Tan,et al.  Graph Matching Iris Image Blocks with Local Binary Pattern , 2006, ICB.

[47]  Tsuhan Chen,et al.  Learning patch correspondences for improved viewpoint invariant face recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Dennis Gabor,et al.  Theory of communication , 1946 .

[50]  M. Bastiaans,et al.  Gabor's signal expansion and the Zak transform. , 1994, Applied optics.

[51]  Sébastien Marcel,et al.  Haar Local Binary Pattern Feature for Fast Illumination Invariant Face Detection , 2009, BMVC.

[52]  Dao-Qing Dai,et al.  Face Recognition Using Dual-Tree Complex Wavelet Features , 2009, IEEE Transactions on Image Processing.

[53]  Matti Pietikäinen,et al.  Combining appearance and motion for face and gender recognition from videos , 2009, Pattern Recognit..

[54]  Jean-Philippe Thiran,et al.  Face detection with boosted Gaussian features , 2007, Pattern Recognit..

[55]  Ben J. A. Kröse,et al.  Efficient Greedy Learning of Gaussian Mixture Models , 2003, Neural Computation.

[56]  T. Strohmer,et al.  Gabor Analysis and Algorithms , 2012 .

[57]  Ammad Ali,et al.  Face Recognition with Local Binary Patterns , 2012 .

[58]  Matti Pietikäinen,et al.  IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2009, TPAMI-2008-09-0620 1 WLD: A Robust Local Image Descriptor , 2022 .

[59]  Lior Wolf,et al.  Using Biologically Inspired Features for Face Processing , 2007, International Journal of Computer Vision.

[60]  Stan Z. Li,et al.  Jensen-Shannon boosting learning for object recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[61]  Wen Gao,et al.  Locally Assembled Binary (LAB) feature with feature-centric cascade for fast and accurate face detection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Josef Kittler,et al.  Image Feature Localization by Multiple Hypothesis Testing of Gabor Features , 2008, IEEE Transactions on Image Processing.

[63]  Sébastien Marcel,et al.  Parts-Based Face Verification Using Local Frequency Bands , 2009, ICB.

[64]  Jian-Gang Wang,et al.  Age categorization via ECOC with fused gabor and LBP features , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[65]  Joni-Kristian Kämäräinen,et al.  Feature representation and discrimination based on Gaussian mixture model probability densities - Practices and algorithms , 2006, Pattern Recognit..

[66]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[67]  Yunde Jia,et al.  Face recognition with local steerable phase feature , 2006, Pattern Recognit. Lett..

[68]  Yunhong Wang,et al.  Faceprint: Fusion of Local Features for 3D Face Recognition , 2009, ICB.

[69]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Berk Gökberk,et al.  Learning the best subset of local features for face recognition , 2007, Pattern Recognit..

[71]  Jian Sun,et al.  Face Alignment Via Component-Based Discriminative Search , 2008, ECCV.

[72]  Matti Pietikäinen,et al.  Face recognition based on the appearance of local regions , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[73]  John Daugman,et al.  High Confidence Visual Recognition of Persons by a Test of Statistical Independence , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Sébastien Marcel,et al.  Face Authentication Using Adapted Local Binary Pattern Histograms , 2006, ECCV.

[75]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[76]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[78]  Xihong Wu,et al.  Boosting Local Binary Pattern (LBP)-Based Face Recognition , 2004, SINOBIOMETRICS.

[79]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[80]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[81]  M. Pietikäinen,et al.  A discriminative feature space for detecting and recognizing faces , 2004, CVPR 2004.

[82]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[83]  Bingpeng Ma,et al.  Robust Head Pose Estimation Using LGBP , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[84]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[85]  Xianghua Xie,et al.  Handbook of Texture Analysis , 2008 .

[86]  Andrea Lagorio,et al.  On the Use of SIFT Features for Face Authentication , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[87]  Shaogang Gong,et al.  Sparse Multiscale Local Binary Patterns , 2006, BMVC.

[88]  Wen Gao,et al.  Histogram of Gabor Phase Patterns (HGPP): A Novel Object Representation Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[89]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[90]  Osamu Yamaguchi,et al.  Facial feature localization using weighted vector concentration approach , 2010, Image Vis. Comput..

[91]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[92]  Qiang Ji,et al.  A Comparative Study of Local Matching Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[93]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[94]  Stan Z. Li,et al.  Learning to Fuse 3D+2D Based Face Recognition at Both Feature and Decision Levels , 2005, AMFG.

[95]  Timo Ahonen,et al.  Recognition of blurred faces using Local Phase Quantization , 2008, 2008 19th International Conference on Pattern Recognition.

[96]  Chuan Yi Tang,et al.  A 2.|E|-Bit Distributed Algorithm for the Directed Euler Trail Problem , 1993, Inf. Process. Lett..

[97]  G. Granlund.  In search of a general picture processing operator , 1978 .

[98]  Changyin Sun,et al.  Gender Classification Based on Boosting Local Binary Pattern , 2006, ISNN.

[99]  Alex Pentland,et al.  A Bayesian similarity measure for direct image matching , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[100]  Stan Z. Li,et al.  Shape localization based on statistical method using extended local binary pattern , 2004, Third International Conference on Image and Graphics (ICIG'04).

[101]  Bhavin J. Shastri,et al.  Face recognition using localized features based on non-negative sparse coding , 2006, Machine Vision and Applications.

[102]  Hong Chen,et al.  A Hierarchical Compositional Model for Face Representation and Sketching , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[103]  Rama Chellappa,et al.  Discriminant Analysis for Recognition of Human Face Images (Invited Paper) , 1997, AVBPA.

[104]  Shu Liao,et al.  Dominant Local Binary Patterns for Texture Classification , 2009, IEEE Transactions on Image Processing.