Dictionary Integration Using 3D Morphable Face Models for Pose-Invariant Collaborative-Representation-Based Classification

The paper presents a dictionary integration algorithm using 3D morphable face models (3DMM) for pose-invariant collaborative-representation-based face classification. To this end, we first fit a 3DMM to the 2D face images of a dictionary to reconstruct the 3D shape and texture of each image. The 3D faces are used to render a number of virtual 2D face images with arbitrary pose variations to augment the training data, by merging the original and rendered virtual samples to create an extended dictionary. Second, to reduce the information redundancy of the extended dictionary and improve the sparsity of reconstruction coefficient vectors using collaborative-representation-based classification (CRC), we exploit an on-line class elimination scheme to optimise the extended dictionary by identifying the training samples of the most representative classes for a given query. The final goal is to perform pose-invariant face classification using the proposed dictionary integration method and the on-line pruning strategy under the CRC framework. Experimental results obtained for a set of well-known face data sets demonstrate the merits of the proposed method, especially its robustness to pose variations.

[1]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[2]  David Zhang,et al.  Relaxed collaborative representation for pattern classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Wen Gao,et al.  Locally Linear Regression for Pose-Invariant Face Recognition , 2007, IEEE Transactions on Image Processing.

[5]  Hong Liu,et al.  Using the original and 'symmetrical face' training samples to perform representation based two-step face recognition , 2013, Pattern Recognit..

[6]  Zhenhua Guo,et al.  Face recognition by sparse discriminant analysis via joint L2, 1-norm minimization , 2014, Pattern Recognit..

[7]  Jian Yang,et al.  Regularized Robust Coding for Face Recognition , 2012, IEEE Transactions on Image Processing.

[8]  Yanqing Guo,et al.  Synthesis linear classifier based analysis dictionary learning for pattern classification , 2017, Neurocomputing.

[9]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[11]  William J. Christmas,et al.  A facial symmetry prior for improved illumination fitting of 3D morphable model , 2013, 2013 International Conference on Biometrics (ICB).

[12]  Jian Yang,et al.  Why can LDA be performed in PCA transformed space? , 2003, Pattern Recognit..

[13]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[14]  Patrik Huber,et al.  Wavelet Reduced Support Vector Regression for Efficient and Robust Head Pose Estimation , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[15]  Jian Yang,et al.  A New Discriminative Sparse Representation Method for Robust Face Recognition via $l_{2}$ Regularization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[17]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jun Guo,et al.  Lighting-aware face frontalization for unconstrained face recognition , 2017, Pattern Recognit..

[19]  Mohammed Bennamoun,et al.  Linear Regression for Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  William J. Christmas,et al.  Cascaded Collaborative Regression for Robust Facial Landmark Detection Trained Using a Mixture of Synthetic and Real Images With Dynamic Weighting , 2015, IEEE Transactions on Image Processing.

[21]  William J. Christmas,et al.  Automatic face annotation by multilinear AAM with Missing Values , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[22]  Xiaogang Wang,et al.  Multi-View Perceptron: a Deep Model for Learning Face Identity and View Representations , 2014, NIPS.

[23]  Daijin Kim,et al.  Tensor-Based AAM with Continuous Variation Estimation: Application to Variation-Robust Face Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  David Zhang,et al.  Collaborative Representation based Classification for Face Recognition , 2012, ArXiv.

[25]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[26]  William J. Christmas,et al.  Fitting 3D Morphable Face Models using local features , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[27]  Dong Xu,et al.  Multilinear Discriminant Analysis for Face Recognition , 2007, IEEE Transactions on Image Processing.

[28]  William J. Christmas,et al.  3D Morphable Face Models and Their Applications , 2016, AMDO.

[29]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[30]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[31]  Xiaojun Wu,et al.  Dynamic dictionary optimization for sparse-representation-based face classification using local difference images , 2017, Inf. Sci..

[32]  Jun Guo,et al.  Extended SRC: Undersampled Face Recognition via Intraclass Variant Dictionary , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  William J. Christmas,et al.  Gaussian mixture 3D morphable face model , 2018, Pattern Recognit..

[34]  Thomas Vetter,et al.  Synthesis of Novel Views from a Single Face Image , 1998, International Journal of Computer Vision.

[35]  William J. Christmas,et al.  Random Cascaded-Regression Copse for Robust Facial Landmark Detection , 2015, IEEE Signal Processing Letters.

[36]  David Zhang,et al.  Using the idea of the sparse representation to perform coarse-to-fine face recognition , 2013, Inf. Sci..

[37]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[38]  Du-Sik Park,et al.  Rotating your face using multi-task deep neural network , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[40]  Josef Kittler,et al.  Face Detection, Bounding Box Aggregation and Pose Estimation for Robust Facial Landmark Localisation in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[42]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Jian Yang,et al.  Integrating Conventional and Inverse Representation for Face Recognition , 2014, IEEE Transactions on Cybernetics.

[44]  Allen Y. Yang,et al.  Fast ℓ1-minimization algorithms and an application in robust face recognition: A review , 2010, 2010 IEEE International Conference on Image Processing.

[45]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[46]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[47]  Lei Zhang,et al.  Metaface learning for sparse representation based face recognition , 2010, 2010 IEEE International Conference on Image Processing.

[48]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[49]  Se-Young Oh,et al.  Simple hybrid classifier for face recognition with adaptively generated virtual data , 2002, Pattern Recognit. Lett..

[50]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[51]  Sanghamitra Bandyopadhyay,et al.  A Symmetry Based Face Detection Technique , 2007 .

[52]  Chien-Hsing Chou,et al.  Application of associative memory in human face detection , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[53]  Honggang Zhang,et al.  Local Sparse Representation Based Classification , 2010, 2010 20th International Conference on Pattern Recognition.

[54]  Xibei Yang,et al.  Towards multi-scale fuzzy sparse discriminant analysis using local third-order tensor model of face images , 2016, Neurocomputing.

[55]  Jian Yang,et al.  A Two-Phase Test Sample Sparse Representation Method for Use With Face Recognition , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[56]  Mubarak Shah,et al.  Face Recognition in Movie Trailers via Mean Sequence Sparse Representation-Based Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Jing-Yu Yang,et al.  Progressive sparse representation-based classification using local discrete cosine transform evaluation for image recognition , 2015, J. Electronic Imaging.

[59]  Eli Saber,et al.  Frontal-view face detection and facial feature extraction using color, shape and symmetry based cost functions , 1998, Pattern Recognit. Lett..

[60]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[61]  Tena Rodriguez,et al.  3D face modelling for 2D+3D face recognition , 2007 .

[62]  Harry Shum,et al.  Statistical Learning of Multi-view Face Detection , 2002, ECCV.

[63]  Chi-Ho Chan,et al.  Face Recognition Using a Unified 3D Morphable Model , 2016, ECCV.

[64]  William J. Christmas,et al.  Dynamic Attention-Controlled Cascaded Shape Regression Exploiting Training Data Augmentation and Fuzzy-Set Sample Weighting , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).