Generalized Augmentation of Multiple Kernels

Kernel combination is meant to improve the performance of single kernels and avoid the difficulty of kernel selection. The most common way of combining kernels is to compute their weighted sum. Usually, the kernels are assumed to exist in independent empirical feature spaces and therefore were combined without considering their relationships. To take these relationships into consideration in kernel combination, we propose the generalized augmentation kernel which is extended by all the single kernels considering their correlations. The generalized augmentation kernel, unlike the weighted sum kernel, does not need to find out the weight of each kernel, and also would not suffer from information loss due to the average of kernels. In the experiments, we observe that the generalized augmentation kernel usually can achieve better performances than other combination methods that do not consider relationship between kernels.

[1]  Gustavo Camps-Valls,et al.  Composite kernels for hyperspectral image classification , 2006, IEEE Geoscience and Remote Sensing Letters.

[2]  Javier M. Moguerza,et al.  Combining Kernel Information for Support Vector Classification , 2004, Multiple Classifier Systems.

[3]  Charu C. Aggarwal,et al.  [7] A. Asuncion and D. J. Newman. UCI Machine Learning Repository , 2008 .

[4]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[5]  Anil K. Jain,et al.  Classifier design with Parzen Windows , 1988 .

[6]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[7]  Nello Cristianini,et al.  8 On Kernel Target Alignment , .

[8]  Koby Crammer,et al.  Kernel Design Using Boosting , 2002, NIPS.

[9]  Murat Dundar,et al.  A fast iterative algorithm for fisher discriminant using heterogeneous kernels , 2004, ICML.

[10]  Josef Kittler,et al.  Combining Multiple Kernels by Augmenting the Kernel Matrix , 2010, MCS.

[11]  Kristin P. Bennett,et al.  MARK: a boosting algorithm for heterogeneous kernel models , 2002, KDD.

[12]  Chin-Teng Lin,et al.  Support-vector-based fuzzy neural network for pattern classification , 2006, IEEE Transactions on Fuzzy Systems.

[13]  Olivier Bousquet,et al.  On the Complexity of Learning the Kernel Matrix , 2002, NIPS.

[14]  Alexander J. Smola,et al.  Learning the Kernel with Hyperkernels , 2005, J. Mach. Learn. Res..

[15]  Ivor W. Tsang,et al.  Efficient hyperkernel learning using second-order cone programming , 2006, IEEE Trans. Neural Networks.

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[18]  Charles A. Micchelli,et al.  Learning the Kernel Function via Regularization , 2005, J. Mach. Learn. Res..

[19]  Javier M. Moguerza,et al.  Improving Support Vector Classification via the Combination of Multiple Sources of Information , 2004, SSPR/SPR.

[20]  Wan-Jui Lee,et al.  Kernel Combination Versus Classifier Combination , 2007, MCS.