Sparse PCA by iterative elimination algorithm

In this paper we proposed an iterative elimination algorithm for sparse principal component analysis. It recursively eliminates variables according to certain criterion that aims to minimize the loss of explained variance, and reconsiders the sparse principal component analysis problem until the desired sparsity is achieved. Two criteria, the approximated minimal variance loss (AMVL) criterion and the minimal absolute value criterion, are proposed to select the variables eliminated in each iteration. Deflation techniques are discussed for multiple principal components computation. The effectiveness is illustrated by both simulations on synthetic data and applications on real data.

[1]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[2]  Jorge Cadima Departamento de Matematica.  Loading and correlations in the interpretation of principle compenents , 1995 .

[3]  I. Jolliffe,et al.  A Modified Principal Component Technique Based on the LASSO , 2003 .

[4]  T. Poggio,et al.  Multiclass cancer diagnosis using tumor gene expression signatures , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Yurii Nesterov,et al.  Generalized Power Method for Sparse Principal Component Analysis , 2008, J. Mach. Learn. Res..

[6]  Lester W. Mackey,et al.  Deflation Methods for Sparse PCA , 2008, NIPS.

[7]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[8]  Jianhua Z. Huang,et al.  Sparse principal component analysis via regularized low rank matrix approximation , 2008 .

[9]  Gert R. G. Lanckriet,et al.  A D.C. Programming Approach to the Sparse Generalized Eigenvalue Problem , 2009, 0901.1504.

[10]  Amnon Shashua,et al.  Nonnegative Sparse PCA , 2006, NIPS.

[11]  Gene H. Golub,et al.  Matrix computations , 1983 .

[12]  Gert R. G. Lanckriet,et al.  Sparse eigen methods by D.C. programming , 2007, ICML '07.

[13]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[14]  Shai Avidan,et al.  Spectral Bounds for Sparse PCA: Exact and Greedy Algorithms , 2005, NIPS.

[15]  Alexandre d'Aspremont,et al.  Optimal Solutions for Sparse Principal Component Analysis , 2007, J. Mach. Learn. Res..

[16]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, NIPS 2004.

[17]  Gert R. G. Lanckriet,et al.  A majorization-minimization approach to the sparse generalized eigenvalue problem , 2011, Machine Learning.

[18]  J. N. R. Jeffers,et al.  Two Case Studies in the Application of Principal Component Analysis , 1967 .