Video-based face recognition via joint sparse representation

In video-based face recognition, a key challenge is in exploiting the extra information available in a video; e.g., face, body, and motion identity cues. In addition, different video sequences of the same subject may contain variations in resolution, illumination, pose, and facial expressions. These variations contribute to the challenges in designing an effective video-based face-recognition algorithm. We propose a novel multivariate sparse representation method for video-to-video face recognition. Our method simultaneously takes into account correlations as well as coupling information among the video frames. Our method jointly represents all the video data by a sparse linear combination of training data. In addition, we modify our model so that it is robust in the presence of noise and occlusion. Furthermore, we kernelize the algorithm to handle the non-linearities present in video data. Numerous experiments using unconstrained video sequences show that our method is effective and performs significantly better than many state-of-the-art video-based face recognition algorithms in the literature.

[1]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[5]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[6]  Rama Chellappa,et al.  3D Facial Pose Tracking in Uncalibrated Videos , 2005, PReMI.

[7]  David J. Kriegman,et al.  Visual tracking and recognition using probabilistic appearance manifolds , 2005, Comput. Vis. Image Underst..

[8]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[9]  Arun Ross,et al.  Handbook of Multibiometrics , 2006, The Kluwer international series on biometrics.

[10]  Roberto Cipolla,et al.  Face Recognition from Video Using the Generic Shape-Illumination Manifold , 2006, ECCV.

[11]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[12]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[13]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Wen Gao,et al.  Manifold-Manifold Distance with application to face recognition based on image set , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Rama Chellappa,et al.  Statistical analysis on Stiefel and Grassmann manifolds with applications in computer vision , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[17]  Massimo Tistarelli,et al.  Handbook of Remote Biometrics: for Surveillance and Security , 2009 .

[18]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Rama Chellappa,et al.  Handbook of Remote Biometrics , 2009, Advances in Pattern Recognition.

[20]  Bruce A. Draper,et al.  Overview of the Multiple Biometrics Grand Challenge , 2009, ICB.

[21]  Rama Chellappa,et al.  Video Précis: Highlighting Diverse Aspects of Videos , 2010, IEEE Transactions on Multimedia.

[22]  Hakan Cevikalp,et al.  Face recognition based on image sets , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Rama Chellappa,et al.  Statistical Computations on Grassmann and Stiefel Manifolds for Image and Video-Based Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[25]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[26]  Julianne H. Ayyad,et al.  Recognizing people from dynamic and static faces and bodies: Dissecting identity with a fusion approach , 2010, Vision Research.

[27]  Ajmal S. Mian,et al.  Sparse approximated nearest points for image set classification , 2011, CVPR 2011.

[28]  Rama Chellappa,et al.  Secure and Robust Iris Recognition Using Random Projections and Sparse Representations , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Rama Chellappa,et al.  Joint Sparsity-Based Robust Multimodal Biometrics Recognition , 2012, ECCV Workshops.

[30]  Rama Chellappa,et al.  Dictionary-Based Face Recognition from Video , 2012, ECCV.

[31]  Rama Chellappa,et al.  Remote identification of faces: Problems, prospects, and progress , 2012, Pattern Recognit. Lett..

[32]  Rama Chellappa,et al.  Dictionary-Based Face Recognition Under Variable Lighting and Pose , 2012, IEEE Transactions on Information Forensics and Security.