Sparse Kernel Canonical Correlation Analysis via $\ell_1$-regularization

Abstract—Canonical correlation analysis (CCA) is a multi-variate statistical technique for finding the linear relationshipbetween two sets of variables. The kernel generalization ofCCA named kernel CCA has been proposed to find nonlinearrelations between data sets. Despite the wide usage of CCAand kernel CCA, they have one common limitation that is thelack of sparsity in their solution. In this paper, we considersparse kernel CCA and propose a novel sparse kernel CCAalgorithm (SKCCA). Our algorithm is based on a relationshipbetween kernel CCA and least squares. Sparsity of the dualtransformations is introduced by penalizing the ‘ 1 -norm of dualvectors. Experiments demonstrate that our algorithm not onlyperforms well in computing sparse dual transformations butalso can alleviate the over-fitting problem of kernel CCA.Index Terms—canonical correlation analysis, kernel, sparsity I. I NTRODUCTION T HE description of relationship between two sets ofvariables has long been an interesting topic to manyresearchers. Canonical correlation analysis (CCA) [10] isa multivariate statistical technique for finding the linearrelationship between two sets of variables. It seeks a lineartransformation for each of the two sets of variables in away that the projected variables in the transformed spaceare maximally correlated. In recent years, CCA has beensuccessfully applied in various areas, including genomicdata analysis [19], [20] and bilingual analysis [18], whereresearchers can measure multiple sets of variables on a singlesubject. For instance, DNA copy number variations, geneexpression, and single nucleotide polymorphism (SNP) datamight all be available on a common set of patient samples.Since CCA only consider linear transformation of theoriginal variables, it fails to capture nonlinear relations. How-ever, in a wide range of practical problems linear relationsmay not be adequate for studying relation among variables.Detecting nonlinear relations among data is important anduseful in modern data analysis, especially when dealingwith data that are not in the form of vectors, such as textdocuments, images, microarray data and so on. A naturalextension, therefore, is to explore and exploit nonlinear rela-tions among data. Among nonlinear extensions of CCA, onemost frequently used approach is the kernel generalization