A Novel Method to Solve the Separation Problem of LDA

Linear discriminant analysis (LDA) is one of the most classical linear projection techniques for feature extraction, widely used in kinds of fields. Classical LDA is contributed to finding an optimal projection subspace that can maximize the between-class scatter and minimize the average within-class scatter of each class. However, the class separation problem always exists and classical LDA can not guarantee that the within-class scatter of each class get its minimum. In this paper, we proposed the k-classifiers method, which can reduce every within-class scatter of classes respectively and alleviate the class separation problem. This method will be applied in LDA and Norm LDA and achieve significant improvement. Extensive experiments performed on MNIST data sets demonstrate the effectiveness of k-classifiers.

[1]  Aleix M. Martínez,et al.  Bayes Optimality in Linear Discriminant Analysis , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Dong Xu,et al.  Trace Ratio vs. Ratio Trace for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  M. Schervish.  Linear discrimination for three known normal populations , 1984 .

[4]  Dit-Yan Yeung,et al.  Worst-Case Linear Discriminant Analysis , 2010, NIPS.

[5]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Chao Yao,et al.  Approximative Bayes optimality linear discriminant analysis for Chinese handwriting character recognition , 2016, Neurocomputing.

[7]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Nong Sang,et al.  Max-min distance analysis by making a uniform distribution of class centers for dimensionality reduction , 2014, Neurocomputing.

[9]  Dacheng Tao,et al.  Max-Min Distance Analysis by Using Sequential SDP Relaxation for Dimension Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[11]  C. R. Rao,et al.  The Utilization of Multiple Measurements in Problems of Biological Classification , 1948 .

[12]  Kuldip K. Paliwal,et al.  Linear discriminant analysis for the small sample size problem: an overview , 2014, International Journal of Machine Learning and Cybernetics.