Sparse Uncorrelated Linear Discriminant Analysis

In this paper, we develop a novel approach for sparse uncorrelated linear discriminant analysis (ULDA). Our proposal is based on characterization of all solutions of the generalized ULDA. We incorporate sparsity into the ULDA transformation by seeking the solution with minimum l1-norm from all minimum dimension solutions of the generalized ULDA. The problem is then formulated as a l1-minimization problem and is solved by accelerated linearized Bregman method. Experiments on high-dimensional gene expression data demonstrate that our approach not only computes extremely sparse solutions but also performs well in classification. Experimental results also show that our approach can help for data visualization in low-dimensional space.

[1]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[2]  Shiqian Ma,et al.  Accelerated Linearized Bregman Method , 2011, J. Sci. Comput..

[3]  Jieping Ye,et al.  Least squares linear discriminant analysis , 2007, ICML '07.

[4]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[5]  Wotao Yin,et al.  Analysis and Generalizations of the Linearized Bregman Method , 2010, SIAM J. Imaging Sci..

[6]  Yeung Sam Hung,et al.  Characterization of All Solutions for Undersampled Uncorrelated Linear Discriminant Analysis Problems , 2011, SIAM J. Matrix Anal. Appl..

[7]  R. Tibshirani,et al.  Penalized classification using Fisher's linear discriminant , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[8]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[9]  Jing-Yu Yang,et al.  Face recognition based on the uncorrelated discriminant transformation , 2001, Pattern Recognit..

[10]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[11]  Michael K Ng,et al.  On sparse Fisher discriminant method for microarray data analysis , 2007, Bioinformation.

[12]  Jieping Ye,et al.  Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems , 2005, J. Mach. Learn. Res..

[13]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[14]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[15]  Murat Dundar,et al.  Sparse Fisher Discriminant Analysis for Computer Aided Detection , 2005, SDM.

[16]  Jian-Feng Cai,et al.  Convergence of the linearized Bregman iteration for ℓ1-norm minimization , 2009, Math. Comput..

[17]  Jieping Ye,et al.  Feature extraction via generalized uncorrelated linear discriminant analysis , 2004, ICML.

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Xihong Lin,et al.  Sparse linear discriminant analysis for simultaneous testing for the significance of a gene set/pathway and gene selection , 2009, Bioinform..

[20]  Trevor J. Hastie,et al.  Sparse Discriminant Analysis , 2011, Technometrics.

[21]  Shai Avidan,et al.  Generalized spectral bounds for sparse LDA , 2006, ICML.

[22]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[23]  Jieping Ye,et al.  Feature Reduction via Generalized Uncorrelated Linear Discriminant Analysis , 2006, IEEE Transactions on Knowledge and Data Engineering.

[24]  Haesun Park,et al.  Generalizing discriminant analysis using the generalized singular value decomposition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .