Kernel Hebbian Algorithm for Iterative Kernel Principal Component Analysis

A new method for performing a kernel principal component analysis is proposed. By kernelizing the generalized Hebbian algorithm, one can iteratively estimate the principal components in a reproducing kernel Hilbert space with only linear order memory complexity. The derivation of the method and preliminary applications in image hyperresolution are presented. In addition, we discuss the extension of the method to the online learning of kernel principal components.

[1]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[2]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[3]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[4]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[5]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[6]  Frank M. Candocia.  A UNIFIED SUPERRESOLUTION APPROACH FOR OPTICAL AND SYNTHETIC APERTURE RADAR IMAGES , 1998 .

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[9]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[10]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[11]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[13]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[14]  Ivor W. Tsang,et al.  Finding the pre-images in kernel principal component analysis , 2002 .