Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

The nonlinear discriminant function obtained using a minimum squared error cost function can be shown to be directly related to the nonlinear Fisher discriminant (NFD). With the squared error cost function, the orthogonal least squares (OLS) algorithm can be used to find a parsimonious description of the nonlinear discriminant function. Two simple classification techniques will be introduced and tested on a number of real and artificial data sets. The results show that the new classification technique can often perform favourably compared with other state of the art classification techniques.

[1]  Stephen A. Billings,et al.  On-line Supervised Adaptive Training Using Radial Basis Function Networks , 1996, Neural Networks.

[2]  Dingli Yu,et al.  A Recursive Orthogonal Least Squares Algorithm for Training RBF Networks , 1997, Neural Processing Letters.

[3]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[4]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[5]  S. A. Billings,et al.  Structure selective updating for nonlinear models and radial basis function neural networks , 1998 .

[6]  Michael E. Tipping.  The Relevance Vector Machine , 1999, NIPS.

[7]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[8]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[9]  Dingli Yu,et al.  Selecting radial basis function network centers with recursive orthogonal least squares training , 2000, IEEE Trans. Neural Networks Learn. Syst..

[10]  Stephen A. Billings,et al.  Adaptive model selection and estimation for nonlinear systems using a sliding data window , 1995, Signal Process..

[11]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[12]  Bart Kosko,et al.  Neural networks for signal processing , 1992 .

[13]  Sheng Chen,et al.  Regularized orthogonal least squares algorithm for constructing radial basis function networks , 1996 .

[14]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[17]  K. Lang,et al.  Learning to tell two spirals apart , 1988 .

[18]  Stephen A. Billings,et al.  On-line Structure Detection and Parameter Estimation with Exponential Windowing for Nonlinear Systems , 1996, Eur. J. Control.

[19]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .