Nonlinear Feature Extraction Using Fisher Criterion

In this paper the problem of nonlinear feature extraction based on the optimization of the Fisher criterion is analyzed. A new nonlinear feature extraction method is proposed. The method does not make use of numerical algorithms and it has an analytical (closed-form) solution. Moreover, no assumptions on the class probability distribution functions are imposed. The proposed method is applied to some standard pattern recognition problems and compared with other classical methodologies already proposed in the literature. The performance of the proposed method turned out to be superior when compared with the other methods studied.

[1]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  B. Everitt,et al.  Statistical methods for rates and proportions , 1973 .

[6]  Lawrence B. Holder,et al.  Current And Future Trends In Feature Selection And Extraction For Classification Problems , 2005, Int. J. Pattern Recognit. Artif. Intell..

[7]  Jonny Eriksson,et al.  Feature reduction for classification of multidimensional data , 2000, Pattern Recognit..

[8]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[9]  N. Campbell.  CANONICAL VARIATE ANALYSIS—A GENERAL MODEL FORMULATION , 1984 .

[10]  Ling-Hwei Chen,et al.  A new approach for audio classification and segmentation using Gabor wavelets and Fisher linear discriminator , 2005, Int. J. Pattern Recognit. Artif. Intell..

[11]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[12]  Johan A. K. Suykens,et al.  Bayesian Framework for Least-Squares Support Vector Machine Classifiers, Gaussian Processes, and Kernel Fisher Discriminant Analysis , 2002, Neural Computation.

[13]  David J. Kriegman,et al.  Recognition using class specific linear projection , 1997 .

[14]  Bernhard Schölkopf,et al.  An improved training algorithm for kernel Fisher discriminants , 2001, AISTATS.

[15]  R. Fisher.  THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[16]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Javier Ruiz-del-Solar,et al.  On the Generalization of Kernel Machines , 2002, SVM.

[18]  Trevor Hastie,et al.  Flexible discriminant and mixture models , 2000 .

[19]  Jian Huang,et al.  Two-step single parameter regularization fisher discriminant method for face recognition , 2006, Int. J. Pattern Recognit. Artif. Intell..

[20]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[21]  Feng Zhang.  Nonlinear feature extraction and dimension reduction by polygonal principal curves , 2006, Int. J. Pattern Recognit. Artif. Intell..

[22]  Andreas G. Andreou,et al.  On Generalizations of Linear Discriminant Analysis , 1996 .

[23]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[24]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[25]  David J. Marchette,et al.  Recursive dimensionality reduction using Fisher's linear discriminant , 1998, Pattern Recognit..

[26]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[27]  C. R. Rao,et al.  The Utilization of Multiple Measurements in Problems of Biological Classification , 1948 .

[28]  Gunnar Rätsch,et al.  A Mathematical Programming Approach to the Kernel Fisher Algorithm , 2000, NIPS.

[29]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[30]  R. Tibshirani,et al.  Flexible Discriminant Analysis by Optimal Scoring , 1994 .

[31]  Richard S. Johannes,et al.  Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus , 1988 .

[32]  Kevin Baker,et al.  Classification of radar returns from the ionosphere using neural networks , 1989 .

[33]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.