Learning geometric combinations of Gaussian kernels with alternating Quasi-Newton algorithm

We propose a novel algorithm for learning a geometric com- bination of Gaussian kernel jointly with a SVM classifier. This problem is the product counterpart of MKL, with restriction to Gaussian kernels. Our algorithm finds a local solution by alternating a Quasi-Newton gradi- ent descent over the kernels and a classical SVM solver over the instances. We show promising results on well known data sets which suggest the soundness of the approach.

[1]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[2]  Andrew Zisserman,et al.  Multiple kernels for object detection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  Christian Igel,et al.  Maximum Likelihood Model Selection for 1-Norm Soft Margin SVMs with Multiple Parameters , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Patrick Gallinari,et al.  SGD-QN: Careful Quasi-Newton Stochastic Gradient Descent , 2009, J. Mach. Learn. Res..

[5]  Manik Varma,et al.  More generality in efficient multiple kernel learning , 2009, ICML '09.

[6]  Jason Weston,et al.  Fast Kernel Classifiers with Online and Active Learning , 2005, J. Mach. Learn. Res..

[7]  O. Chapelle Second order optimization of kernel parameters , 2008 .