Weakly Supervised Learning on Pre-image Problem in Kernel Methods

This paper presents a novel alternative approach, namely weakly supervised learning (WSL), to learn the pre-image of a feature vector in the feature space induced by a kernel. It is known that the exact pre-image may typically seldom exist, since the input space and the feature space are not isomorphic in general, and an approximate solution is required in past. The proposed WSL, however, would find an appropriate rather than only a purely approximate solution. WSL is able to involve some weakly supervised prior knowledge into the study of pre-image. The prior knowledge is weak and no class label of the sample is required, providing only information of positive class and negative class which should properly depend on applications. The proposed algorithm is demonstrated on kernel principal component analysis (KPCA) with application to illumination normalization and image denoising on faces. Evaluations of the performance of the proposed algorithm show notable improvement as comparing with some well-known existing approaches

[1]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[2]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[3]  S. Li,et al.  1 D-LDA versus 2 D-LDA : When Is Vector-based Linear Discriminant Analysis Better than Matrix-based ? , 2008 .

[4]  Ivor W. Tsang,et al.  The pre-image problem in kernel methods , 2003, IEEE Transactions on Neural Networks.

[5]  Jian-Huang Lai,et al.  On Constrained Sparse Matrix Factorization , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Bernhard Schölkopf,et al.  Learning to Find Pre-Images , 2003, NIPS.

[7]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[8]  Jian-Huang Lai,et al.  The Solution Space for Fisher Discriminant Analysis and the Uniqueness Under Constraints , 2004, SINOBIOMETRICS.

[9]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[10]  Jian-Huang Lai,et al.  Fast Calculation for Fisher Criteria in Small Sample Size Problem , 2004, SINOBIOMETRICS.

[11]  Jian-Huang Lai,et al.  Face illumination normalization on large and small scale features , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Ran He,et al.  Reducing Impact of Inaccurate User Feedback in Face Retrieval , 2008, 2008 Chinese Conference on Pattern Recognition.

[14]  Jian-Huang Lai,et al.  Regularized Locality Preserving Learning of Pre-Image Problem in Kernel Principal Component Analysis , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[15]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[16]  Jian-Huang Lai,et al.  GA-fisher: a new LDA-based face recognition algorithm with selection of principal components , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).