Fractional-order embedding canonical correlation analysis and its applications to multi-view dimensionality reduction and recognition

Due to the noise disturbance and limited number of training samples, within-set and between-set sample covariance matrices in canonical correlation analysis (CCA) usually deviate from the true ones. In this paper, we re-estimate within-set and between-set covariance matrices to reduce the negative effect of this deviation. Specifically, we use the idea of fractional order to respectively correct the eigenvalues and singular values in the corresponding sample covariance matrices, and then construct fractional-order within-set and between-set scatter matrices which can obviously alleviate the problem of the deviation. On this basis, a new approach is proposed to reduce the dimensionality of multi-view data for classification tasks, called fractional-order embedding canonical correlation analysis (FECCA). The proposed method is evaluated on various handwritten numeral, face and object recognition problems. Extensive experimental results on the CENPARMI, UCI, AT&T, AR, and COIL-20 databases show that FECCA is very effective and obviously outperforms the existing joint dimensionality reduction or feature extraction methods in terms of classification accuracy. Moreover, its improvements for recognition rates are statistically significant on most cases below the significance level 0.05. A new method, namely FECCA, is presented for multi-view dimensionality reduction.Fractional-order within-set and between-set scatter matrices are constructed by sample spectrum modeling.The extracted features by FECCA have strong discriminant power for recognition.Experimental results show FECCA has better recognition rates than the existing joint feature extraction methods.

[1]  Ping Xu,et al.  Modified linear discriminant analysis approaches for classification of high-dimensional microarray data , 2009, Comput. Stat. Data Anal..

[2]  Yan Liu,et al.  A new method of feature fusion and its application in image recognition , 2005, Pattern Recognit..

[3]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[4]  Hujun Bao,et al.  A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[6]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[8]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Sham M. Kakade,et al.  Multi-view Regression Via Canonical Correlation Analysis , 2007, COLT.

[10]  Pheng-Ann Heng,et al.  A Novel Feature Fusion Method Based on Partial Least Squares Regression , 2005, ICAPR.

[11]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[12]  Jintao Zhang,et al.  Inductive multi-task learning with multiple view data , 2012, KDD.

[13]  Horst Bischof,et al.  Appearance models based on kernel canonical correlation analysis , 2003, Pattern Recognit..

[14]  Anastasia Lykou,et al.  Sparse CCA using a Lasso with positivity constraints , 2010, Comput. Stat. Data Anal..

[15]  B. Silverman,et al.  Canonical correlation analysis when the data are curves. , 1993 .

[16]  Daoqiang Zhang,et al.  Multi-view dimensionality reduction via canonical random correlation analysis , 2015, Frontiers of Computer Science.

[17]  Allan Aasbjerg Nielsen,et al.  Multiset canonical correlations analysis and multispectral, truly multitemporal remote sensing data , 2002, IEEE Trans. Image Process..

[18]  I. Jolliffe.  Principal Component Analysis , 2002 .

[19]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[20]  Jian Yang,et al.  From classifiers to discriminators: A nearest neighbor rule induced discriminant analysis , 2011, Pattern Recognit..

[21]  Sham M. Kakade,et al.  Multi-view clustering via canonical correlation analysis , 2009, ICML '09.

[22]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[23]  Jidong Zhao,et al.  Locality sensitive semi-supervised feature selection , 2008, Neurocomputing.

[24]  Jieping Ye,et al.  Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Yi-Fei Pu,et al.  Fractional Differential Mask: A Fractional Differential-Based Approach for Multiscale Texture Enhancement , 2010, IEEE Transactions on Image Processing.

[26]  Xiaoyang Tan,et al.  Fractional order singular value decomposition representation for face recognition , 2008, Pattern Recognit..

[27]  Gene H. Golub,et al.  Matrix computations , 1983 .

[28]  Farzin Deravi,et al.  A review of speech-based bimodal recognition , 2002, IEEE Trans. Multim..

[29]  Bart De Moor,et al.  Meta-clustering of gene expression data and literature-based information , 2003, SKDD.

[30]  Pheng-Ann Heng,et al.  A theorem on the generalized canonical projective vectors , 2005, Pattern Recognit..

[31]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[32]  Konstantinos N. Plataniotis,et al.  Regularized discriminant analysis for the small sample size problem in face recognition , 2003, Pattern Recognit. Lett..

[33]  Malcolm Slaney,et al.  FaceSync: A Linear Operator for Measuring Synchronization of Video Facial Images and Audio Tracks , 2000, NIPS.

[34]  Jian Yang,et al.  KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  A. Murat Tekalp,et al.  Audiovisual Synchronization and Fusion Using Canonical Correlation Analysis , 2007, IEEE Transactions on Multimedia.

[37]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[38]  Xiaohong Chen,et al.  A unified dimensionality reduction framework for semi-paired and semi-supervised multi-view data , 2012, Pattern Recognit..

[39]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[40]  Philip S. Yu,et al.  A General Model for Multiple View Unsupervised Learning , 2008, SDM.

[41]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[42]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[43]  Ching Y. Suen,et al.  Rejection Criteria and Pairwise Discrimination of Handwritten Numerals Based on Structural Features , 1999, Pattern Analysis & Applications.

[44]  Wei Pan,et al.  An Adaptable-Multilayer Fractional Fourier Transform Approach for Image Registration , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Alain Baccini,et al.  CCA: An R Package to Extend Canonical Correlation Analysis , 2008 .

[46]  Feiping Nie,et al.  Multiple view semi-supervised dimensionality reduction , 2010, Pattern Recognit..

[47]  Luuk J. Spreeuwers,et al.  A Bootstrap Approach to Eigenvalue Correction , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[48]  Quan-Sen Sun,et al.  Discriminative learning of multiset integrated canonical correlation analysis for feature fusion , 2012, 2012 15th International Conference on Information Fusion.

[49]  Qiang Zhou,et al.  A novel multiset integrated canonical correlation analysis framework and its application in feature fusion , 2011, Pattern Recognit..

[50]  Raymond Veldhuis,et al.  Eigenvalue correction results in face recognition , 2008 .

[51]  M. Barker,et al.  Partial least squares for discrimination , 2003 .

[52]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[53]  John Shawe-Taylor,et al.  Sparse canonical correlation analysis , 2009, Machine Learning.

[54]  Hu Zhong,et al.  HANDWRITTEN DIGIT RECOGNITION BASED ON MULTI-CLASSIFIER COMBINATION , 1999 .

[55]  Songcan Chen,et al.  Locality preserving CCA with applications to data visualization and pose estimation , 2007, Image Vis. Comput..

[56]  Hirokazu Kameoka,et al.  SemiCCA: Efficient Semi-supervised Learning of Canonical Correlations , 2010, 2010 20th International Conference on Pattern Recognition.