Trace norm regularized canonical correlation analysis

Canonical correlation analysis(CCA) is a popular technique that works for finding the correlation between two sets of variables. However, CCA faces the problem of small sample size in dealing with high dimensional data. Several approaches have been proposed to overcome this issue, but the resulting transformation matrix fails to extract shared structures among data samples. In this paper, we propose trace norm regularized CCA(SRCCA) that not only tackles the problem of small sample size but also uncover the underlying structures between target classes. Specifically, our formulation characterizes the intrinsic dimensionality of a transformation matrix owing to the appealing property of trace norm. Evaluations over public data sets deliver the effectiveness of our algorithm.

[1]  Daniela M Witten,et al.  Extensions of Sparse Canonical Correlation Analysis with Applications to Genomic Data , 2009, Statistical applications in genetics and molecular biology.

[2]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[3]  A. Zwinderman,et al.  Statistical Applications in Genetics and Molecular Biology Quantifying the Association between Gene Expressions and DNA-Markers by Penalized Canonical Correlation Analysis , 2011 .

[4]  Songcan Chen,et al.  Locality preserving CCA with applications to data visualization and pose estimation , 2007, Image Vis. Comput..

[5]  Delin Chu,et al.  Sparse Canonical Correlation Analysis: New Formulation and Algorithm. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[6]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[7]  A. Martínez,et al.  The AR face databasae , 1998 .

[8]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[9]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Michael I. Jordan,et al.  A Probabilistic Interpretation of Canonical Correlation Analysis , 2005 .

[11]  Tommi S. Jaakkola,et al.  Maximum-Margin Matrix Factorization , 2004, NIPS.

[12]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[13]  Hongtao Lu,et al.  Linear discriminant analysis with spectral regularization , 2013, Applied Intelligence.

[14]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[15]  R. Tibshirani,et al.  A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. , 2009, Biostatistics.

[16]  Xiaoyang Tan,et al.  Pattern Recognition , 2016, Communications in Computer and Information Science.

[17]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[18]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[19]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[20]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[21]  Jieping Ye,et al.  A least squares formulation for a class of generalized eigenvalue problems in machine learning , 2009, ICML '09.

[22]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[23]  John Shawe-Taylor,et al.  Sparse canonical correlation analysis , 2009, Machine Learning.

[24]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[25]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[26]  Jieping Ye,et al.  Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Songcan Chen,et al.  Class label versus sample label-based CCA , 2007, Appl. Math. Comput..

[28]  Jieping Ye,et al.  An accelerated gradient method for trace norm minimization , 2009, ICML '09.

[29]  Delin Chu,et al.  Sparse Uncorrelated Linear Discriminant Analysis , 2013, ICML.

[30]  Daoqiang Zhang,et al.  A New Locality-Preserving Canonical Correlation Analysis Algorithm for Multi-View Dimensionality Reduction , 2013, Neural Processing Letters.

[31]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[32]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[33]  Matthijs Douze,et al.  Large-scale image classification with trace-norm regularization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[35]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .