Boosting Multiple Classifiers Constructed by Hybrid Discriminant Analysis

In this paper, a set of hybrid dimension reduction schemes is constructed by unifying principal component analysis (PCA) and linear discriminant analysis (LDA) in a single framework. PCA compensates LDA for singular scatter matrix caused by small set of training samples and increases the effective dimension of the projected subspace. Generalization of hybrid analysis is extended to other discriminant analysis such as multiple discriminant analysis (MDA), and the recent biased discriminant analysis (BDA), and other hybrid pairs. In order to reduce the search time to find the best single classifier, a boosted hybrid analysis is proposed. Our scheme boosts both the individual features as well as a set of weak classifiers. Extensive tests on benchmark and real image databases have shown the superior performance of the boosted hybrid analysis.

[1]  W. Torgerson.  Multidimensional scaling: I. Theory and method , 1952 .

[2]  David G. Stork,et al.  Pattern Classification , 1973 .

[3]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[4]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[5]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[6]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[7]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[8]  Thomas S. Huang,et al.  Small sample learning during multimedia retrieval using BiasMap , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  I. Jolliffe.  Principal Component Analysis , 2002 .

[11]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[12]  Qi Tian,et al.  Parameterized discriminant analysis for image classification , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[13]  Qi Tian,et al.  Self-supervised learning based on discriminative nonlinear features for image classification , 2005, Pattern Recognit..