On the Lagrangian biduality of sparsity minimization problems

We present a novel primal-dual analysis on a class of NP-hard sparsity minimization problems to provide new interpretations for their well known convex relaxations. We show that the Lagrangian bidual (i.e., the Lagrangian dual of the Lagrangian dual) of the sparsity minimization problems can be used to derive interesting convex relaxations: the bidual of the ℓ<sub>0</sub>-minimization problem is ℓ<sub>1</sub>-minimization; and the bidual of ℓ<sub>0,1</sub>-minimization for enforcing group sparsity on structured data is ℓ<sub>1,∞</sub>-minimization problem. Intuitions from the bidual-based relaxation are used to introduce a new family of relaxations for the group sparsity minimization problem.

[1]  Marc Teboulle,et al.  Convex approximations to sparse PCA via Lagrangian duality , 2011, Oper. Res. Lett..

[2]  Michael Elad,et al.  Image Sequence Denoising via Sparse and Redundant Representations , 2009, IEEE Transactions on Image Processing.

[3]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Stéphane Chrétien.  An alternating l1 approach to the compressed sensing problem , 2007 .

[5]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Richard G. Baraniuk,et al.  Compressive imaging for video representation and coding , 2006 .

[7]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[8]  Francis R. Bach,et al.  Structured sparsity-inducing norms through submodular functions , 2010, NIPS.

[9]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[10]  MaYi,et al.  Dense error correction via l1-minimization , 2010 .

[11]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[12]  Sundeep Rangan,et al.  Necessary and Sufficient Conditions for Sparsity Pattern Recovery , 2008, IEEE Transactions on Information Theory.

[13]  I. Loris.  On the performance of algorithms for the minimization of ℓ1-penalized functionals , 2007, 0710.4082.

[14]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[15]  René Vidal,et al.  Robust classification using structured sparse representation , 2011, CVPR 2011.

[16]  E.J. Candes.  Compressive Sampling , 2022 .

[17]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[18]  John Wright,et al.  Dense Error Correction Via $\ell^1$-Minimization , 2010, IEEE Transactions on Information Theory.

[19]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[20]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[21]  A. Juditsky,et al.  Verifiable Conditions of L1-recovery of Sparse Signals with Sign Restriction , 2009 .

[22]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[23]  David L. Donoho,et al.  Neighborly Polytopes And Sparse Solution Of Underdetermined Linear Equations , 2005 .

[24]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[25]  Volkan Cevher,et al.  Sparse Signal Recovery Using Markov Random Fields , 2008, NIPS.

[26]  A. Martínez,et al.  The AR face databasae , 1998 .

[27]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[28]  Stéphane Chrétien,et al.  An Alternating $l_1$ Approach to the Compressed Sensing Problem , 2008, IEEE Signal Processing Letters.

[29]  Aleix M. Martinez,et al.  The AR face database , 1998 .