Geometry and homotopy for l 1 sparse representations

We explore the geometry ofl1 sparse representations in both the noiseless (Basis Pursuit) and noisy (Basis Pursuit De-N oising) case using a homotopy method. We will see that the concept of the basi s vertexc, which has unit inner product with active basis vectors, is a useful geometric concept, both for visualization and for algorithm construction. We derive an explicit homotopy continuation algorithm and find that this method has interesting parallels with the Polytope Faces Pursuit a lgorithm for the noiseless case. Numerical results confirm the operation of t he algorithm.

[1]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[2]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[3]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[4]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[5]  Joseph F. Murray,et al.  Dictionary Learning Algorithms for Sparse Representation , 2003, Neural Computation.

[6]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[7]  Mato Baotic,et al.  Multi-Parametric Toolbox (MPT) , 2004, HSCC.

[8]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[9]  R. Tibshirani,et al.  REJOINDER TO "LEAST ANGLE REGRESSION" BY EFRON ET AL. , 2004, math/0406474.

[10]  M. H. Wright.  The interior-point revolution in optimization: History, recent developments, and lasting consequences , 2004 .

[11]  J. Tropp.  JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION , 2004 .

[12]  David L. Donoho,et al.  Neighborly Polytopes And Sparse Solution Of Underdetermined Linear Equations , 2005 .

[13]  Berwin A. Turlach,et al.  On algorithms for solving least squares problems under an L1 penalty or an L1 constraint , 2005 .

[14]  Dmitry M. Malioutov,et al.  Homotopy continuation for sparse signal representation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[15]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[16]  Mark D. Plumbley.  Recovery of Sparse Representations by Polytope Faces Pursuit , 2006, ICA.

[17]  Mark D. Plumbley.  On Polar Polytopes and the Recovery of Sparse Representations , 2007, IEEE Trans. Inf. Theory.