A DC Programming Approach for Sparse Eigenvalue Problem

We investigate the sparse eigenvalue problem which arises in various fields such as machine learning and statistics. Unlike standard approaches relying on approximation of the l0-norm, we work with an equivalent reformulation of the problem at hand as a DC program. Our starting point is the eigenvalue problem to which a constraint for sparsity requirement is added. The obtained problem is first formulated as a mixed integer program, and exact penalty techniques are used to equivalently transform the resulting problem into a DC program, whose solution is assumed by a customized DCA. Computational results for sparse principal component analysis are reported, which show the usefulness of our approach that compares favourably with some related standard methods using approximation of the l0-norm.

[1]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[2]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[3]  Jorge Cadima Departamento de Matematica.  Loading and correlations in the interpretation of principle compenents , 1995 .

[4]  Gert R. G. Lanckriet,et al.  Sparse eigen methods by D.C. programming , 2007, ICML '07.

[5]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[6]  W. Gander,et al.  A D.C. OPTIMIZATION ALGORITHM FOR SOLVING THE TRUST-REGION SUBPROBLEM∗ , 1998 .

[7]  J. N. R. Jeffers,et al.  Two Case Studies in the Application of Principal Component Analysis , 1967 .

[8]  Le Thi Hoai An,et al.  Combining DCA (DC Algorithms) and interior point techniques for large-scale nonconvex quadratic programming , 2008, Optim. Methods Softw..

[9]  Bernhard Schölkopf,et al.  Use of the Zero-Norm with Linear Models and Kernel Methods , 2003, J. Mach. Learn. Res..

[10]  Le Thi Hoai An,et al.  A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem , 1998, SIAM J. Optim..

[11]  Y. Saad.  Projection and deflation method for partial pole assignment in linear state feedback , 1988 .

[12]  Le Thi Hoai An,et al.  Large-Scale Molecular Optimization from Distance Matrices by a D.C. Optimization Approach , 2003, SIAM J. Optim..

[13]  Lester W. Mackey,et al.  Deflation Methods for Sparse PCA , 2008, NIPS.

[14]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[15]  P. White.  The Computation of Eigenvalues and Eigenvectors of a Matrix , 1958 .

[16]  Attila Kuba,et al.  A Benchmark Evaluation of Large-Scale Optimization Approaches to Binary Tomography , 2006, DGCI.

[17]  Shai Avidan,et al.  Spectral Bounds for Sparse PCA: Exact and Greedy Algorithms , 2005, NIPS.

[18]  Alexandre d'Aspremont,et al.  Optimal Solutions for Sparse Principal Component Analysis , 2007, J. Mach. Learn. Res..

[19]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, NIPS 2004.

[20]  L. Ghaoui.  On the Quality of a Semidefinite Programming Bound for Sparse Principal Component Analysis , 2006, math/0601448.

[21]  I. Jolliffe,et al.  A Modified Principal Component Technique Based on the LASSO , 2003 .