Image Set Classification Using Holistic Multiple Order Statistics Features and Localized Multi-kernel Metric Learning

This paper presents a new approach for image set classification, where each training and testing example contains a set of image instances of an object captured from varying viewpoints or under varying illuminations. While a number of image set classification methods have been proposed in recent years, most of them model each image set as a single linear subspace or mixture of linear subspaces, which may lose some discriminative information for classification. To address this, we propose exploring multiple order statistics as features of image sets, and develop a localized multi-kernel metric learning (LMKML) algorithm to effectively combine different order statistics information for classification. Our method achieves the state-of-the-art performance on four widely used databases including the Honda/UCSD, CMU Mobo, and Youtube face datasets, and the ETH-80 object dataset.

[1]  Rama Chellappa,et al.  Dictionary-Based Face Recognition from Video , 2012, ECCV.

[2]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Ken-ichi Maeda,et al.  Face recognition using temporal image sequence , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[5]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[6]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[7]  Ajmal S. Mian,et al.  Face Recognition Using Sparse Approximated Nearest Points between Image Sets , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Ajmal S. Mian,et al.  Sparse approximated nearest points for image set classification , 2011, CVPR 2011.

[9]  Chiou-Shann Fuh,et al.  Multiple Kernel Learning for Dimensionality Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Trevor Darrell,et al.  Face recognition with image sets using manifold density divergence , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Shaogang Gong,et al.  Person re-identification by probabilistic relative distance comparison , 2011, CVPR 2011.

[12]  Josef Kittler,et al.  Learning Discriminative Canonical Correlations for Object Recognition with Image Sets , 2006, ECCV.

[13]  Alan C. Bovik,et al.  No-reference image blur assessment using multiscale gradient , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[14]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[15]  Vladimir Pavlovic,et al.  Face tracking and recognition with visual constraints in real-world videos , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Yunde Jia,et al.  Discriminant Clustering Embedding for Face Recognition with Image Sets , 2007, ACCV.

[17]  Xiaoming Chen,et al.  Margin Preserving Projection for Image Set Based Face Recognition , 2011, ICONIP.

[18]  Qionghai Dai,et al.  Weighted Subspace Distance and Its Applications to Object Recognition and Retrieval With Image Sets , 2009, IEEE Signal Processing Letters.

[19]  Likun Huang,et al.  Face recognition based on image sets , 2014 .

[20]  Kristen Grauman,et al.  Kernelized locality-sensitive hashing for scalable image search , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Ruiping Wang,et al.  Manifold Discriminant Analysis , 2009, CVPR.

[22]  Matti Pietikäinen,et al.  From still image to video-based face recognition: an experimental analysis , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[23]  Qi Tian,et al.  Multiple Kernel Learning with High Order Kernels , 2010, 2010 20th International Conference on Pattern Recognition.

[24]  Bin Zhao,et al.  Multiple Kernel Clustering , 2009, SDM.

[25]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[26]  Prateek Jain,et al.  Fast image search for learned metrics , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Brian C. Lovell,et al.  Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching , 2011, CVPR 2011.

[28]  Shiguang Shan,et al.  Image sets alignment for Video-Based Face Recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Masayuki Mukunoki,et al.  Set Based Discriminative Ranking for Recognition , 2012, ECCV.

[30]  Brian C. Lovell,et al.  Face Recognition from Still Images to Video Sequences: A Local-Feature-Based Framework , 2011, EURASIP J. Image Video Process..

[31]  Wen Gao,et al.  Manifold-Manifold Distance with application to face recognition based on image set , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Ethem Alpaydin,et al.  Localized multiple kernel learning , 2008, ICML '08.

[33]  David J. Kriegman,et al.  Video-based face recognition using probabilistic appearance manifolds , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[34]  Ivor W. Tsang,et al.  Domain Transfer Multiple Kernel Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jun Wang,et al.  Metric Learning with Multiple Kernels , 2011, NIPS.

[36]  Bruce A. Draper,et al.  Image-set matching using a geodesic distance and cohort normalization , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[37]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[38]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[39]  Dit-Yan Yeung,et al.  Locally Linear Models on Face Appearance Manifolds with Application to Dual-Subspace Based Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).