Oriented Discriminant Analysis (ODA)

Linear discriminant analysis (LDA) has been an active topic of research during the last century. However, the existing algorithms have several limitations when applied to visual data. LDA is only optimal for gaussian distributed classes with equal covariance matrices and just classes-1 features can be extracted. On the other hand, LDA does not scale well to high dimensional data (over-fitting) and it does not necessarily minimize the classification error. In this paper, we introduce Oriented Discriminant Analysis (ODA), a LDA extension which can overcome these drawbacks. Three main novelties are proposed: • An optimal dimensionality reduction which maximizes the KullbackLiebler divergencebetweenclasses is proposed. This allowsus tomodel class covariances and to extract more than classes-1 features. • Several covariance approximations are introduced to improve classification in the small sample case. • A linear time iterative majorization method is introduced in order to find a local optimal solution. Several synthetic and real experiments on face recognition are reported 1 .

[1]  Max Welling,et al.  Extreme Components Analysis , 2003, NIPS.

[2]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[3]  Andreas G. Andreou,et al.  Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition , 1998, Speech Commun..

[4]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[5]  Takeo Kanade,et al.  Multimodal oriented discriminant analysis , 2005, ICML.

[6]  Shaogang Gong,et al.  Recognising trajectories of facial identities using kernel discriminant analysis , 2003, Image and Vision Computing.

[7]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[8]  Kohji Fukunaga,et al.  Introduction to Statistical Pattern Recognition-Second Edition , 1990 .

[9]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[10]  David G. Stork,et al.  Pattern Classification , 1973 .

[11]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[13]  Jan de Leeuw,et al.  Block-relaxation Algorithms in Statistics , 1994 .

[14]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[15]  Y. Freund,et al.  Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By , 2000 .

[16]  George Saon,et al.  Maximum likelihood discriminant feature spaces , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[17]  Henk A. L. Kiers.  Maximization of sums of quotients of quadratic forms and some generalizations , 1995 .

[18]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[19]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[20]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[21]  Wenyi Zhao,et al.  Discriminant component analysis for face recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.