Model Selection for Gaussian Kernel PCA Denoising

We propose kernel parallel analysis (kPA) for automatic kernel scale and model order selection in Gaussian kernel principal component analysis (KPCA). Parallel analysis is based on a permutation test for covariance and has previously been applied for model order selection in linear PCA, we here augment the procedure to also tune the Gaussian kernel scale of radial basis function based KPCA. We evaluate kPA for denoising of simulated data and the U.S. postal data set of handwritten digits. We find that kPA outperforms other heuristics to choose the model order and kernel scale in terms of signal-to-noise ratio of the denoised data.

[1]  Pedro M. Valero-Mora,et al.  Determining the Number of Factors to Retain in EFA: An easy-to-use computer program for carrying out Parallel Analysis , 2007 .

[2]  Kurt Stadlthanner,et al.  KPCA denoising and the pre-image problem revisited , 2008, Digit. Signal Process..

[3]  Paul Golder,et al.  The Guttman-Kaiser Criterion as a Predictor of the Number of Common Factors , 1982 .

[4]  J. Horn A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[5]  Ivor W. Tsang,et al.  The pre-image problem in kernel methods , 2003, IEEE Transactions on Neural Networks.

[6]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Johan A. K. Suykens,et al.  Kernel Component Analysis Using an Epsilon-Insensitive Robust Loss Function , 2008, IEEE Transactions on Neural Networks.

[8]  D. Calvetti,et al.  Iterative methods for the computation of a few eigenvalues of a large symmetric matrix , 1996 .

[9]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[10]  Renaud Keriven,et al.  Normalization and preimage problem in gaussian kernel PCA , 2008, 2008 15th IEEE International Conference on Image Processing.

[11]  Guillermo Sapiro,et al.  Connecting the Out-of-Sample and Pre-Image Problems in Kernel Methods , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  André Beauducel,et al.  Problems with parallel analysis in data sets with oblique simple structure , 2001 .

[13]  L. K. Hansen,et al.  Generalizable Patterns in Neuroimaging: How Many Principal Components? , 1999, NeuroImage.

[14]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.