Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria

We derive a class of computationally inexpensive linear dimension reduction criteria by introducing a weighted variant of the well-known K-class Fisher criterion associated with linear discriminant analysis (LDA). It can be seen that LDA weights contributions of individual class pairs according to the Euclidean distance of the respective class means. We generalize upon LDA by introducing a different weighting function.

[1]  H. P. Decell,et al.  Feature combinations and the divergence criterion , 1977 .

[2]  C. R. Rao,et al.  The Utilization of Multiple Measurements in Problems of Biological Classification , 1948 .

[3]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[4]  R. Fisher.  THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[5]  Andreas G. Andreou,et al.  Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition , 1998, Speech Commun..

[6]  Dean M. Young,et al.  A formulation and comparison of two linear feature selection techniques applicable to statistical classification , 1984, Pattern Recognit..

[7]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[8]  Ljubomir J. Buturovic.  Toward Bayes-Optimal Linear Dimension Reduction , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  M. Loog.  Approximate Pairwise Accuracy Criteria for Multiclass Linear Dimension Reduction: Generalisations of the Fisher Criterion , 1999 .

[10]  Robert P. W. Duin,et al.  Multi-class linear feature extraction by nonlinear PCA , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[11]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[12]  N. Campbell.  CANONICAL VARIATE ANALYSIS—A GENERAL MODEL FORMULATION , 1984 .

[13]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[14]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.