Improvement of the kernel minimum squared error model for fast feature extraction

The kernel minimum squared error (KMSE) expresses the feature extractor as a linear combination of all the training samples in the high-dimensional kernel space. To extract a feature from a sample, KMSE should calculate as many kernel functions as the training samples. Thus, the computational efficiency of the KMSE-based feature extraction procedure is inversely proportional to the size of the training sample set. In this paper, we propose an efficient kernel minimum squared error (EKMSE) model for two-class classification. The proposed EKMSE expresses each feature extractor as a linear combination of nodes, which are a small portion of the training samples. To extract a feature from a sample, EKMSE only needs to calculate as many kernel functions as the nodes. As the nodes are commonly much fewer than the training samples, EKMSE is much faster than KMSE in feature extraction. The EKMSE can achieve the same training accuracy as the standard KMSE. Also, EKMSE avoids the overfitting problem. We implement the EKMSE model using two algorithms. Experimental results show the feasibility of the EKMSE model.

[1]  David Zhang,et al.  A fast kernel-based nonlinear discriminant analysis for multi-class problems , 2006, Pattern Recognit..

[2]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[3]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[4]  Qin Li,et al.  Fast kernel Fisher discriminant analysis via approximating the kernel principal component analysis , 2011, Neurocomputing.

[5]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[6]  Jian Yang,et al.  A reformative kernel Fisher discriminant analysis , 2004, Pattern Recognit..

[7]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[8]  Jing-Yu Yang,et al.  An efficient kernel-based nonlinear regression method for two-class classification , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[9]  Jianguo Sun,et al.  Recursive reduced least squares support vector regression , 2009, Pattern Recognit..

[10]  Haibo Zhang,et al.  A fast method of feature extraction for kernel MSE , 2011, Neurocomputing.

[11]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[12]  Feiping Nie,et al.  A general kernelization framework for learning algorithms based on kernel PCA , 2010, Neurocomputing.

[13]  Haibo Zhang,et al.  Pruning least objective contribution in KMSE , 2011, Neurocomputing.

[14]  Kwang In Kim,et al.  Face recognition using kernel principal component analysis , 2002, IEEE Signal Processing Letters.

[15]  Qi Zhu.  Reformative nonlinear feature extraction using kernel MSE , 2010, Neurocomputing.

[16]  Xuegong Zhang,et al.  Kernel MSE algorithm: a unified framework for KFD, LS-SVM and KRR , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[17]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[18]  Jian Yang,et al.  A reformative kernel Fisher discriminant algorithm and its application to face recognition , 2006, Neurocomputing.