Bundle methods for dual atomic pursuit

The aim of structured optimization is to assemble a solution, using a given set of (possibly uncountably infinite) atoms, to fit a model to data. A two-stage algorithm based on gauge duality and bundle method is proposed. The first stage discovers the optimal atomic support for the primal problem by solving a sequence of approximations of the dual problem using a bundle-type method. The second stage recovers the approximate primal solution using the atoms discovered in the first stage. The overall approach leads to implementable and efficient algorithms for large problems.

[1]  Dmitriy Drusvyatskiy,et al.  Foundations of Gauge and Perspective Duality , 2017, SIAM J. Optim..

[2]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[3]  Joel A. Tropp,et al.  Living on the edge: phase transitions in convex programs with random data , 2013, 1303.6672.

[4]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[5]  J. Y. Bello Cruz,et al.  Level bundle-like algorithms for convex optimization , 2013, Journal of Global Optimization.

[6]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[7]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[8]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[9]  Michael P. Friedlander,et al.  Gauge Optimization and Duality , 2013, SIAM J. Optim..

[10]  Franz Rendl,et al.  A Spectral Bundle Method for Semidefinite Programming , 1999, SIAM J. Optim..

[11]  F. Bonsall.  A GENERAL ATOMIC DECOMPOSITION THEOREM AND BANACH'S CLOSED RANGE THEOREM , 1991 .

[12]  Yurii Nesterov,et al.  New variants of bundle methods , 1995, Math. Program..

[13]  Per Olov Lindberg,et al.  A descent proximal level bundle method for convex nondifferentiable optimization , 1995, Oper. Res. Lett..

[14]  Michael P. Friedlander,et al.  Low-Rank Spectral Optimization via Gauge Duality , 2015, SIAM J. Sci. Comput..

[15]  Weiyu Xu,et al.  Necessary and sufficient conditions for success of the nuclear norm heuristic for rank minimization , 2008, 2008 47th IEEE Conference on Decision and Control.

[16]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[17]  Volkan Cevher,et al.  An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity , 2019, SIAM J. Optim..

[18]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[19]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .