Novel Fisher discriminant classifiers

At the present, several applications need to classify high dimensional points belonging to highly unbalanced classes. Unfortunately, when the training set cardinality is small compared to the data dimensionality (''small sample size'' problem) the classification performance of several well-known classifiers strongly decreases. Similarly, the classification accuracy of several discriminative methods decreases when non-linearly separable, and unbalanced, classes are treated. In this paper we firstly survey state of the art methods that employ improved versions of Linear Discriminant Analysis (LDA) to deal with the above mentioned problems; secondly, we propose a family of classifiers based on the Fisher subspace estimation, which efficiently deal with the small sample size problem, non-linearly separable classes, and unbalanced classes. The promising results obtained by the proposed techniques on benchmark datasets and the comparison with state of the art predictors show the efficacy of the proposed techniques.

[1]  Jieping Ye,et al.  An optimization criterion for generalized discriminant analysis on undersampled problems , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  D. Paul ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL , 2007 .

[3]  C. T. Ng,et al.  Measures of distance between probability distributions , 1989 .

[4]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[5]  J. Friedman Regularized Discriminant Analysis , 1989 .

[6]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[7]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[8]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[9]  J. Fox Applied Regression Analysis, Linear Models, and Related Methods , 1997 .

[10]  HansenPer Christian The truncated SVD as a method for regularization , 1987 .

[11]  G. Valentini,et al.  DDAG K-TIPCAC: an ensemble method for protein subcellular localization , 2010 .

[12]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[13]  G. Medioni,et al.  Tensor Voting : Theory and Applications , 2000 .

[14]  G. McLachlan Discriminant Analysis and Statistical Pattern Recognition , 1992 .

[15]  Ming-Hsuan Yang,et al.  Extended isomap for pattern classification , 2002, AAAI/IAAI.

[16]  R. Chellappa,et al.  Subspace Linear Discriminant Analysis for Face Recognition , 1999 .

[17]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[18]  Qi Tian,et al.  Image Classification By The Foley-Sammon Transform , 1986 .

[19]  Ming-Hsuan Yang,et al.  Face recognition using extended isomap , 2002, Proceedings. International Conference on Image Processing.

[20]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[21]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[22]  Konstantinos N. Plataniotis,et al.  Regularized discriminant analysis for the small sample size problem in face recognition , 2003, Pattern Recognit. Lett..

[23]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[24]  Hanqing Lu,et al.  Improving kernel Fisher discriminant analysis for face recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Alessandro Rozza,et al.  O-IPCAC and its Application to EEG Classification , 2010, WAPA.

[26]  Konstantinos N. Plataniotis,et al.  Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition , 2005, Pattern Recognit. Lett..

[27]  Jian Yang,et al.  Essence of kernel Fisher discriminant: KPCA plus LDA , 2004, Pattern Recognit..

[28]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[29]  M. Saunders,et al.  Towards a Generalized Singular Value Decomposition , 1981 .

[30]  Robert P. W. Duin,et al.  Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[32]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[33]  I. Johnstone,et al.  Sparse Principal Components Analysis , 2009, 0901.4392.

[34]  Jing-Yu Yang,et al.  Optimal discriminant plane for a small number of samples and design method of classifier on the plane , 1991, Pattern Recognit..

[35]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[36]  Xu-Yan Tu,et al.  Face recognition using KFD-Isomap , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[37]  Trevor Hastie,et al.  Regularized linear discriminant analysis and its application in microarrays. , 2007, Biostatistics.

[38]  Alessandro Rozza,et al.  Novel IPCA-Based Classifiers and Their Application to Spam Filtering , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[39]  Narendra Ahuja,et al.  Face recognition using kernel eigenfaces , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[40]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[41]  Ming-Hsuan Yang,et al.  Kernel Eigenfaces vs. Kernel Fisherfaces: Face recognition using kernel methods , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[42]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Shaogang Gong,et al.  Recognising trajectories of facial identities using kernel discriminant analysis , 2003, Image Vis. Comput..

[44]  Haesun Park,et al.  Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition , 2003, SIAM J. Matrix Anal. Appl..

[45]  Elena Casiraghi,et al.  A Fully Automated Method for Lung Nodule Detection From Postero-Anterior Chest Radiographs , 2006, IEEE Transactions on Medical Imaging.

[46]  Mohammad T. Manzuri Shalmani,et al.  Heteroscedastic Multilinear Discriminant Analysis for Face Recognition , 2010, 2010 20th International Conference on Pattern Recognition.

[47]  M. Loog Approximate Pairwise Accuracy Criteria for Multiclass Linear Dimension Reduction: Generalisations of the Fisher Criterion , 1999 .

[48]  Haesun Park,et al.  Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[49]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[50]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[51]  Wei Wu,et al.  Orthogonalized discriminant analysis based on generalized singular value decomposition , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[52]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[53]  M. Brand,et al.  Fast low-rank modifications of the thin singular value decomposition , 2006 .

[54]  P J. Phillips,et al.  Subspace Linear Discriminant Analysis for Face Recognition | NIST , 2008 .

[55]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[56]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[57]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[58]  Wei Wu,et al.  A kernelized discriminant analysis algorithm based on modified generalized singular value decomposition , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[59]  Guodong Guo,et al.  Support Vector Machines Applications , 2014 .

[60]  Gabriele Lombardi,et al.  The Neighbors Voting Algorithm and Its Applications , 2009, Applications of Supervised and Unsupervised Ensemble Methods.

[61]  Santosh S. Vempala,et al.  Isotropic PCA and Affine-Invariant Clustering , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[62]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[63]  Jing-Yu Yang,et al.  Algebraic feature extraction for image recognition based on an optimal discriminant criterion , 1993, Pattern Recognit..

[64]  I. Jolliffe Principal Component Analysis , 2002 .

[65]  Giorgio Valentini,et al.  Support vector machines for candidate nodules classification , 2005, Neurocomputing.