Robust canonical correlation analysis based on L1-norm minimization for feature learning and image recognition

Abstract. Canonical correlation analysis (CCA) is a popular method that has been extensively used in feature learning. In nature, the objective function of CCA is equivalent to minimizing the distance of the paired data, and L2-norm is used as the distance metric. We know that L2-norm-based objective function will emphasize the large distance pairs and de-emphasizes the small distance pairs. To alleviate the aforementioned problems of CCA, we propose an approach named CCA based on L1-norm minimization (CCA-L1) for feature learning. To optimize the objective function, we develop an algorithm that can get a global optimized value. To maintain the distribution and the nonlinear characteristic respectively, we proposed two extensions of CCA-L1. Further, all of the aforementioned three proposed algorithms are extended to deal with multifeature data. The experimental results on an artificial dataset, real-world crop leaf disease dataset, ORL face dataset, and PIE face dataset show that our methods outperform traditional CCA and its variants.

[1]  Fan Zhang,et al.  Discriminative low-rank graph preserving dictionary learning with Schatten-p quasi-norm regularization for image recognition , 2018, Neurocomputing.

[2]  Jing-Yu Yang,et al.  Unsupervised discriminant canonical correlation analysis based on spectral clustering , 2016, Neurocomputing.

[3]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Xuelong Li,et al.  Semisupervised Dimensionality Reduction and Classification Through Virtual Label Regression , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Jing-Yu Yang,et al.  Multiple kernel clustering based on centered kernel alignment , 2014, Pattern Recognit..

[6]  Jian Yang,et al.  L1-Norm Distance Linear Discriminant Analysis Based on an Effective Iterative Algorithm , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Songcan Chen,et al.  Locality preserving CCA with applications to data visualization and pose estimation , 2007, Image Vis. Comput..

[8]  Fernando De la Torre,et al.  A Least-Squares Framework for Component Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[10]  Yan Liu,et al.  A new method of feature fusion and its application in image recognition , 2005, Pattern Recognit..

[11]  Jian Yang,et al.  From classifiers to discriminators: A nearest neighbor rule induced discriminant analysis , 2011, Pattern Recognit..

[12]  Xuelong Li,et al.  Robust Tensor Analysis With L1-Norm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[14]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[15]  Hongyuan Zha,et al.  Adaptive Manifold Learning , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Feiping Nie,et al.  Angle 2DPCA: A New Formulation for 2DPCA , 2018, IEEE Transactions on Cybernetics.

[17]  Shiguang Shan,et al.  Multiset Feature Learning for Highly Imbalanced Data Classification , 2017, AAAI.

[18]  Pheng-Ann Heng,et al.  A theorem on the generalized canonical projective vectors , 2005, Pattern Recognit..

[19]  Dong Yue,et al.  Uncorrelated multi-set feature learning for color face recognition , 2016, Pattern Recognit..

[20]  Feiping Nie,et al.  Cauchy Graph Embedding , 2011, ICML.

[21]  Yong Luo,et al.  Tensor Canonical Correlation Analysis for Multi-View Dimension Reduction , 2015, IEEE Transactions on Knowledge and Data Engineering.

[22]  Haixian Wang,et al.  L1-Norm Kernel Discriminant Analysis Via Bayes Error Bound Optimization for Robust Feature Extraction , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[23]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[24]  Quan-Sen Sun,et al.  Multiset Canonical Correlations Using Globality Preserving Projections With Applications to Feature Extraction and Recognition , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Rong Wang,et al.  Robust 2DPCA With Non-greedy $\ell _{1}$ -Norm Maximization for Image Analysis , 2015, IEEE Transactions on Cybernetics.

[26]  Allan Aasbjerg Nielsen,et al.  Multiset canonical correlations analysis and multispectral, truly multitemporal remote sensing data , 2002, IEEE Trans. Image Process..

[27]  Weifeng Liu,et al.  Canonical correlation analysis networks for two-view image recognition , 2017, Inf. Sci..

[28]  Horst Bischof,et al.  Appearance models based on kernel canonical correlation analysis , 2003, Pattern Recognit..

[29]  Haiping Lu,et al.  Learning Canonical Correlations of Paired Tensor Sets Via Tensor-to-Vector Projection , 2013, IJCAI.

[30]  Panos P. Markopoulos,et al.  Optimal Algorithms for L1-subspace Signal Processing , 2014, IEEE Transactions on Signal Processing.

[31]  Andrzej Cichocki,et al.  Common and Individual Features Analysis: Beyond Canonical Correlation Analysis , 2012, ArXiv.

[32]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[33]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[34]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Liang Wang,et al.  Cross-Modal Subspace Learning via Pairwise Constraints , 2014, IEEE Transactions on Image Processing.

[36]  Xuesong Lu,et al.  Fisher Discriminant Analysis With L1-Norm , 2014, IEEE Transactions on Cybernetics.

[37]  Michael I. Jordan,et al.  A Probabilistic Interpretation of Canonical Correlation Analysis , 2005 .

[38]  Xuelong Li,et al.  L1-Norm-Based 2DPCA , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Jing-Yu Yang,et al.  Face recognition based on the uncorrelated discriminant transformation , 2001, Pattern Recognit..

[40]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[41]  Wen Gao,et al.  Maximal Linear Embedding for Dimensionality Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Karen Livescu,et al.  Nonparametric Canonical Correlation Analysis , 2015, ICML.

[43]  Jiashu Zhang,et al.  Linear Discriminant Analysis Based on L1-Norm Maximization , 2013, IEEE Transactions on Image Processing.

[44]  Stefanos Zafeiriou,et al.  Robust Correlated and Individual Component Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Jiashu Zhang,et al.  Discriminant Locality Preserving Projections Based on L1-Norm Maximization , 2014, IEEE Transactions on Neural Networks and Learning Systems.