Kernel Canonical Correlation Analysis and Least Squares Support Vector Machines

A key idea of nonlinear Support Vector Machines (SVMs) is to map the inputs in a nonlinear way to a high dimensional feature space, while Mercer's condition is applied in order to avoid an explicit expression for the nonlinear mapping. In SVMs for nonlinear classification a large margin classifier is constructed in the feature space. For regression a linear regressor is constructed in the feature space. Other kernel extensions of linear algorithms have been proposed like kernel Principal Component Analysis (PCA) and kernel Fisher Discriminant Analysis. In this paper, we discuss the extension of linear Canonical Correlation Analysis (CCA) to a kernel CCA with application of the Mercer condition. We also discuss links with single output Least Squares SVM (LS-SVM) Regression and Classification.

[1]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[2]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[3]  Brian D. Ripley,et al.  Neural Networks and Related Methods for Classification , 1994 .

[4]  Colin Fyfe,et al.  Kernel and Nonlinear Canonical Correlation Analysis , 2000, IJCNN.

[5]  Christopher K. I. Williams.  Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[6]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[7]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[8]  A. J. Collins,et al.  Introduction To Multivariate Analysis , 1981 .

[9]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[10]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[11]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[12]  W. T. Tutte.  The reconstruction problem in graph theory , 1977 .

[13]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  Johan A. K. Suykens,et al.  Financial time series prediction using least squares support vector machines within the evidence framework , 2001, IEEE Trans. Neural Networks.