Sparse Orthogonal Linear Discriminant Analysis

In this paper, sparse orthogonal linear discriminant analysis (OLDA) is studied. The main contributions of the present work include the following: (i) all minimum Frobenius-norm/dimension solutions of the optimization problem used for establishing OLDA are characterized explicitly; and (ii) this explicit characterization leads to two numerical algorithms for computing a sparse linear transformation for OLDA. The first is based on the gradient flow approach while the second is a sequential linear Bregman method. We experiment with real world datasets to illustrate that the sequential linear Bregman method is much better than the gradient flow approach. The sequential linear Bregman method always achieves comparable classification accuracy with the normal OLDA, satisfactory sparsity and orthogonality, and acceptable CPU times.

[1]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[3]  Pong C. Yuen,et al.  Regularized discriminant analysis and its application to face recognition , 2003, Pattern Recognit..

[4]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[6]  Haesun Park,et al.  Fast Linear Discriminant Analysis using QR Decomposition and Regularization , 2007 .

[7]  S. Dudoit,et al.  Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data , 2002 .

[8]  Michael K Ng,et al.  On sparse Fisher discriminant method for microarray data analysis , 2007, Bioinformation.

[9]  Trevor Hastie,et al.  Regularized linear discriminant analysis and its application in microarrays. , 2007, Biostatistics.

[10]  Haesun Park,et al.  Generalizing discriminant analysis using the generalized singular value decomposition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Delin Chu,et al.  A New and Fast Orthogonal Linear Discriminant Analysis on Undersampled Problems , 2010, SIAM J. Sci. Comput..

[12]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[13]  Xihong Lin,et al.  Sparse linear discriminant analysis for simultaneous testing for the significance of a gene set/pathway and gene selection , 2009, Bioinform..

[14]  Bart De Moor,et al.  A QR-type reduction for computing the SVD of a general matrix product/quotient , 2003, Numerische Mathematik.

[15]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[16]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .