Multi-Order Statistical Descriptors for Real-Time Face Recognition and Object Classification

We propose novel multi-order statistical descriptors which can be used for high speed object classification or face recognition from videos or image sets. We represent each gallery set with a global second-order statistic which captures correlated global variations in all feature directions as well as the common set structure. A lightweight descriptor is then constructed by efficiently compacting the second-order statistic using Cholesky decomposition. We then enrich the descriptor with the first-order statistic of the gallery set to further enhance the representation power. By projecting the descriptor into a low-dimensional discriminant subspace, we obtain further dimensionality reduction, while the discrimination power of the proposed representation is still preserved. Therefore, our method represents a complex image set by a single descriptor having significantly reduced dimensionality. We apply the proposed algorithm on image set and video-based face and periocular biometric identification, object category recognition, and hand gesture recognition. Experiments on six benchmark data sets validate that the proposed method achieves significantly better classification accuracy with lower computational complexity than the existing techniques. The proposed compact representations can be used for real-time object classification and face recognition in videos.

[1]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[2]  Xin Zhao,et al.  Human action recognition based on semi-supervised discriminant analysis with global constraint , 2013, Neurocomputing.

[3]  Pengfei Shi,et al.  Kernel Grassmannian distances and discriminant analysis for face recognition from image sets , 2009, Pattern Recognit. Lett..

[4]  Xinbo Gao,et al.  Graphical Representation for Heterogeneous Face Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Shiguang Shan,et al.  Joint sparse representation for video-based face recognition , 2014, Neurocomputing.

[7]  Lei Zhang,et al.  Face recognition based on regularized nearest points between image sets , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[8]  Nanning Zheng,et al.  Modeling 4D Human-Object Interactions for Joint Event Segmentation, Recognition, and Object Localization , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jian-Huang Lai,et al.  Person Re-Identification by Camera Correlation Aware Feature Augmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Ying Tai,et al.  Nuclear Norm Based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Wen Gao,et al.  Manifold-Manifold Distance with application to face recognition based on image set , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[13]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[14]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[15]  Arif Mahmood,et al.  Hierarchical Sparse Spectral Clustering For Image Set Classification , 2012, BMVC.

[16]  Ajmal S. Mian,et al.  Face Recognition Using Sparse Approximated Nearest Points between Image Sets , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Gang Wang,et al.  Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[19]  Arif Mahmood,et al.  A compact discriminative representation for efficient image-set classification with application to biometric recognition , 2013, 2013 International Conference on Biometrics (ICB).

[20]  Arif Mahmood,et al.  Semi-supervised Spectral Clustering for Image Set Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  David J. Kriegman,et al.  Video-based face recognition using probabilistic appearance manifolds , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  J. Ross Beveridge,et al.  Action classification on product manifolds , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Huimin Ma,et al.  Boundary-aware box refinement for object proposal generation , 2017, Neurocomputing.

[25]  Ajmal S. Mian,et al.  Regularized Least-Squares Coding with Unlabeled Dictionary for Image-Set Based Face Recognition , 2014, 2014 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[26]  Changyin Sun,et al.  Kernel inverse Fisher discriminant analysis for face recognition , 2014, Neurocomputing.

[27]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[28]  Arif Mahmood,et al.  Periocular region-based person identification in the visible, infrared and hyperspectral imagery , 2015, Neurocomputing.

[29]  Mubarak Shah,et al.  Face Recognition in Movie Trailers via Mean Sequence Sparse Representation-Based Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Dacheng Tao,et al.  Trunk-Branch Ensemble Convolutional Neural Networks for Video-Based Face Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Vladimir Pavlovic,et al.  Face tracking and recognition with visual constraints in real-world videos , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Sujing Wang,et al.  Incremental multi-linear discriminant analysis using canonical correlations for action recognition , 2012, Neurocomputing.

[33]  Rama Chellappa,et al.  Guest Editorial Introduction to the Special Issue on Large-Scale Video Analytics for Enhanced Security: Algorithms and Systems , 2017, IEEE Trans. Syst. Man Cybern. Syst..

[34]  Larry S. Davis,et al.  Covariance discriminative learning: A natural and efficient approach to image set classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Brian C. Lovell,et al.  Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching , 2011, CVPR 2011.

[36]  Tae-Kyun Kim,et al.  Tensor Canonical Correlation Analysis for Action Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[38]  Nannan Li,et al.  Video anomaly detection based on a hierarchical activity discovery within spatio-temporal contexts , 2014, Neurocomputing.

[39]  Hyeyoung Park,et al.  Robust recognition of face with partial variations using local features and statistical learning , 2014, Neurocomputing.

[40]  M. Pourahmadi,et al.  Nonparametric estimation of large covariance matrices of longitudinal data , 2003 .

[41]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[42]  David Zhang,et al.  From Point to Set: Extend the Learning of Distance Metrics , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Arif Mahmood,et al.  Sparse Kernel Learning for Image Set Classification , 2014, ACCV.

[44]  Wei Li,et al.  Dimensionality reduction using graph-embedded probability-based semi-supervised discriminant analysis , 2014, Neurocomputing.

[45]  Likun Huang,et al.  Face recognition based on image sets , 2014 .

[46]  Chang-Dong Wang,et al.  Multi-local model image set matching based on domain description , 2014, Pattern Recognit..

[47]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Tae-Kyun Kim,et al.  Boosted manifold principal angles for image set-based recognition , 2007, Pattern Recognit..

[49]  Ruiping Wang,et al.  Manifold Discriminant Analysis , 2009, CVPR.

[50]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[51]  Mohammed Bennamoun,et al.  Iterative deep learning for image set based face and object recognition , 2016, Neurocomputing.

[52]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[53]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[54]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Shiguang Shan,et al.  Sigma Set: A small second order statistical region descriptor , 2009, CVPR.