An improved training algorithm for kernel Fisher discriminants

We present a fast training algorithm for the kernel Fisher discriminant classifier. It uses a greedy approximation technique and has an empirical scaling behavior which improves upon the state of the art by more than an order of magnitude, thus rendering the kernel Fisher algorithm a viable option also for large datasets.

[1]  Bernhard Schölkopf,et al.  Kernel Methods for Extracting Local Image Semantics , 2001 .

[2]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[3]  Bernhard Schölkopf,et al.  Sparse Greedy Matrix Approximation for Machine Learning , 2000, International Conference on Machine Learning.

[4]  Gunnar Rätsch,et al.  Invariant Feature Extraction and Classification in Kernel Spaces , 1999, NIPS.

[5]  Gene H. Golub,et al.  Matrix computations , 1983 .

[6]  R. Vanderbei.  LOQO user's manual — version 3.10 , 1999 .

[7]  Gunnar Rätsch,et al.  A Mathematical Programming Approach to the Kernel Fisher Algorithm , 2000, NIPS.

[8]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[9]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[10]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[11]  Neural Networks for Signal Processing VIII , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[12]  Michael E. Tipping.  The Relevance Vector Machine , 1999, NIPS.

[13]  Alexander J. Smola,et al.  Sparse Greedy Gaussian Process Regression , 2000, NIPS.

[14]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[15]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[16]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[17]  John Shawe-Taylor,et al.  A Column Generation Algorithm For Boosting , 2000, ICML.