Non-sparse Multiple Kernel Learning for Fisher Discriminant Analysis

We consider the problem of learning a linear combination of pre-specified kernel matrices in the Fisher discriminant analysis setting. Existing methods for such a task impose an $\ell_1$ norm regularisation on the kernel weights, which produces sparse solution but may lead to loss of information. In this paper, we propose to use $\ell_2$ norm regularisation instead. The resulting learning problem is formulated as a semi-infinite program and can be solved efficiently. Through experiments on both synthetic data and a very challenging object recognition benchmark, the relative advantages of the proposed method and its $\ell_1$ counterpart are demonstrated, and insights are gained as to how the choice of regularisation norm should be made.