LEARNING WITH INFINITELY MANY KERNELS VIA SEMI-INFINITE PROGRAMMING

Abstract: In recent years, learning methods are desirable because of their reliability and efficiency in real-world problems. We propose a novel method to find infinitely many kernel combinations for learning problems with the help of infinite and semi-infinite optimization regarding all elements in kernel space. This will provide to study variations of combinations of kernels when considering heterogeneous data in real-world applications. Looking at all infinitesimally fine convex combinations of the kernels from the infinite kernel set, the margin is maximized subject to an infinite number of constraints with a compact index set and an additional (Riemann-Stieltjes) integral constraint due to the combinations. After a parametrisation in the space of probability measures it becomes semi-infinite. We analyze the conditions which satisfy the Reduction Ansatz and discuss the type of distribution functions of the kernel coefficients within the structure of the constraints and our bilevel optimization problem.

[1]  Tom M. Apostol,et al.  Mathematical analysis : a modern approach to advanced calculus / Tom M. Apostol , 1957 .

[2]  E. Anderson.  Linear Programming In Infinite Dimensional Spaces , 1970 .

[3]  Rainer Hettich,et al.  Numerische Methoden der Approximation und semi-infiniten Optimierung , 1982 .

[4]  Related Topics,et al.  Parametric Optimization and Related Topics V , 1987 .

[5]  Gautam Appa,et al.  Linear Programming in Infinite-Dimensional Spaces , 1989 .

[6]  M. A. López-Cerdá,et al.  Linear Semi-Infinite Optimization , 1998 .

[7]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[8]  Tong Zhang,et al.  An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods , 2001, AI Mag..

[9]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[10]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[11]  Nello Cristianini,et al.  A statistical framework for genomic data fusion , 2004, Bioinform..

[12]  Gunnar Rätsch,et al.  Large Scale Multiple Kernel Learning , 2006, J. Mach. Learn. Res..

[13]  Kok Lay Teo,et al.  Some properties on quadratic infinite programs of integral type , 2007, Appl. Math. Lett..

[14]  Yves Grandvalet,et al.  More efficiency in multiple kernel learning , 2007, ICML '07.

[15]  J. Shawe-Taylor,et al.  Pattern Analysis for the Prediction of Eukoryatic Pro-peptide Cleavage Sites ? , 2007 .