Face recognition across makeup and plastic surgery from real-world images

Abstract. A study for feature extraction is proposed to handle the problem of facial appearance changes including facial makeup and plastic surgery in face recognition. To extend a face recognition method robust to facial appearance changes, features are individually extracted from facial depth on which facial makeup and plastic surgery have no effect. Then facial depth features are added to facial texture features to perform feature extraction. Accordingly, a three-dimensional (3-D) face is reconstructed from only a single two-dimensional (2-D) frontal image in real-world scenarios. Then the facial depth is extracted from the reconstructed model. Afterward, the dual-tree complex wavelet transform (DT-CWT) is applied to both texture and reconstructed depth images to extract the feature vectors. Finally, the final feature vectors are generated by combining 2-D and 3-D feature vectors, and are then classified by adopting the support vector machine. Promising results have been achieved for makeup-invariant face recognition on two available image databases including YouTube makeup and virtual makeup, and plastic surgery-invariant face recognition on a plastic surgery face database is compared to several state-of-the-art feature extraction methods. Several real-world scenarios are also planned to evaluate the performance of the proposed method on a combination of these three databases with 1102 subjects.

[1]  Patrick J. Flynn,et al.  A sparse representation approach to face matching across plastic surgery , 2012, 2012 IEEE Workshop on the Applications of Computer Vision (WACV).

[2]  Karim Faez,et al.  Unrestricted pose-invariant face recognition by sparse dictionary matrix , 2015, Image Vis. Comput..

[3]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[4]  Ali Moeini,et al.  Real-World and Rapid Face Recognition Toward Pose and Expression Variations via Feature Library Matrix , 2015, IEEE Transactions on Information Forensics and Security.

[5]  Karim Faez,et al.  Real-world gender classification via local Gabor binary pattern and three-dimensional face reconstruction by generic elastic model , 2015, IET Image Process..

[6]  Michele Nappi,et al.  Robust Face Recognition after Plastic Surgery Using Local Region Analysis , 2011, ICIAR.

[7]  S. Ueda,et al.  Influence of Make-up on Facial Recognition , 2010, Perception.

[8]  Yang Lei,et al.  Two-stage sparse representation-based face recognition with reconstructed images , 2014, J. Electronic Imaging.

[9]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[10]  Karim Faez,et al.  Makeup-Invariant Face Recognition by 3D Face: Modeling and Dual-Tree Complex Wavelet Transform from Women's 2D Real-World Images , 2014, 2014 22nd International Conference on Pattern Recognition.

[11]  Ali Moeini,et al.  Makeup-invariant face recognition by combination of Local Binary Pattern and Dual-Tree Complex Wavelet Transform from women's images , 2014, 7'th International Symposium on Telecommunications (IST'2014).

[12]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[13]  G. Rhodes,et al.  Are Average Facial Configurations Attractive Only Because of Their Symmetry? , 1999 .

[14]  Karim Faez,et al.  Makeup-insensitive face recognition by facial depth reconstruction and Gabor filter bank from women's real-world images , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[15]  Dao-Qing Dai,et al.  Face Recognition Using Dual-Tree Complex Wavelet Features , 2009, IEEE Transactions on Image Processing.

[16]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[17]  Karim Faez,et al.  Expression-invariant three-dimensional face reconstruction from a single image by facial expression generic elastic models , 2014, J. Electronic Imaging.

[18]  Shuicheng Yan,et al.  Face Authentication With Makeup Changes , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[20]  Arun Ross,et al.  Automatic facial makeup detection with application in face recognition , 2013, 2013 International Conference on Biometrics (ICB).

[21]  Marios Savvides,et al.  3-D Generic Elastic Models for Fast and Texture Preserving 2-D Novel Pose Synthesis , 2012, IEEE Transactions on Information Forensics and Security.

[22]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Arun Ross,et al.  Can facial cosmetics affect the matching accuracy of face recognition systems? , 2012, 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[24]  Marios Savvides,et al.  Gender and Ethnicity Specific Generic Elastic Models from a Single 2D Image for Novel 2D Pose Face Synthesis and Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Himanshu S. Bhatt,et al.  Recognizing Surgically Altered Face Images Using Multiobjective Evolutionary Algorithm , 2013, IEEE Transactions on Information Forensics and Security.

[26]  Richard Baraniuk,et al.  The Dual-tree Complex Wavelet Transform , 2007 .

[27]  Karim Faez,et al.  Expression-Invariant Face Recognition via 3D Face Reconstruction Using Gabor Filter Bank from a 2D Single Image , 2014, 2014 22nd International Conference on Pattern Recognition.

[28]  Karim Faez,et al.  Unconstrained Head Pose Estimation with Constrained Local Model and Memory Based Particle Filter by 3D Point Distribution Models , 2013 .