Douglas-Rachford Splitting for Pathological Convex Optimization

Despite the vast literature on DRS, there has been very little work analyzing their behavior under pathologies. Most analyses assume a primal solution exists, a dual solution exists, and strong duality holds. When these assumptions are not met, i.e., under pathologies, the theory often breaks down and the empirical performance may degrade significantly. In this paper, we establish that DRS only requires strong duality to work, in the sense that asymptotically iterates are approximately feasible and approximately optimal.

[1]  Heinz H. Bauschke,et al.  Generalized Solutions for the Sum of Two Maximally Monotone Operators , 2013, SIAM J. Control. Optim..

[2]  Michael J. Todd,et al.  Infeasible-start primal-dual methods and infeasibility detectors for nonlinear programming problems , 1999, Math. Program..

[3]  J. Koenderink.  Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[4]  J. Borwein,et al.  Regularizing the Abstract Convex Program , 1981 .

[5]  M. Fortin,et al.  On decomposition - coordination methods using an augmented Lagrangian , 1983 .

[6]  Wotao Yin,et al.  An Envelope for Davis–Yin Splitting and Strict Saddle-Point Avoidance , 2018, J. Optim. Theory Appl..

[7]  Dmitriy Drusvyatskiy,et al.  A note on alternating projections for ill-posed semidefinite feasibility problems , 2017, Math. Program..

[8]  J. Borwein,et al.  Characterizations of optimality without constraint qualification for the abstract convex program , 1982 .

[9]  Ming Yan,et al.  Self Equivalence of the Alternating Direction Method of Multipliers , 2014, 1407.7400.

[10]  Heinz H. Bauschke,et al.  The method of cyclic projections for closed convex sets in Hilbert space , 1997 .

[11]  Patrick L. Combettes,et al.  Monotone operator theory in convex optimization , 2018, Math. Program..

[12]  Gábor Pataki,et al.  Bad Semidefinite Programs: They All Look the Same , 2011, SIAM J. Optim..

[13]  K. S. Kretschmer,et al.  Programmes in Paired Spaces , 1961, Canadian Journal of Mathematics.

[14]  Wotao Yin,et al.  Faster Convergence Rates of Relaxed Peaceman-Rachford and ADMM Under Regularity Assumptions , 2014, Math. Oper. Res..

[15]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[16]  Wotao Yin,et al.  A New Use of Douglas-Rachford Splitting and ADMM for Identifying Infeasible, Unbounded, and Pathological Conic Programs , 2017, ArXiv.

[17]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[18]  Motakuri V. Ramana,et al.  An exact duality theory for semidefinite programming and its complexity implications , 1997, Math. Program..

[19]  Stefano Di Cairano,et al.  Infeasibility detection in alternating direction method of multipliers for convex quadratic programs , 2014, 53rd IEEE Conference on Decision and Control.

[20]  Damek Davis,et al.  Convergence Rate Analysis of Primal-Dual Splitting Schemes , 2014, SIAM J. Optim..

[21]  Maretsugu Yamasaki.  Some generalizations of duality theorems in mathematical programming problems , 1969 .

[22]  Heinz H. Bauschke,et al.  On a result of Pazy concerning the asymptotic behaviour of nonexpansive mappings , 2015, 1505.04129.

[23]  Gábor Pataki,et al.  Sieve-SDP: a simple facial reduction algorithm to preprocess semidefinite programs , 2017, Mathematical Programming Computation.

[24]  Pablo A. Parrilo,et al.  Partial facial reduction: simplified, equivalent SDPs via approximations of the PSD cone , 2014, Math. Program..

[25]  Kim-Chuan Toh,et al.  On the equivalence of inexact proximal ALM and ADMM for a class of convex composite programming , 2018, Math. Program..

[26]  James Renegar,et al.  Computing approximate solutions for convex conic systems of constraints , 2000, Math. Program..

[27]  Dmitriy Drusvyatskiy,et al.  The Many Faces of Degeneracy in Conic Optimization , 2017, Found. Trends Optim..

[28]  Paul Tseng.  Some convex programs without a duality gap , 2009, Math. Program..

[29]  Henry Wolkowicz,et al.  Strong Duality for Semidefinite Programming , 1997, SIAM J. Optim..

[30]  R. Kellogg.  A nonlinear alternating direction method , 1969 .

[31]  Johan Löfberg,et al.  Pre- and Post-Processing Sum-of-Squares Programs in Practice , 2009, IEEE Transactions on Automatic Control.

[32]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[33]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[34]  Dimitri P. Bertsekas,et al.  Convex Optimization Theory , 2009 .

[35]  Heinz H. Bauschke,et al.  Finding best approximation pairs relative to two closed convex sets in Hilbert spaces , 2004, J. Approx. Theory.

[36]  M. Théra,et al.  Generalized sums of monotone operators , 1999 .

[37]  Bruno F. Lourenço,et al.  Solving SDP completely with an interior point oracle , 2021, Optim. Methods Softw..

[38]  Stephen P. Boyd,et al.  OSQP: an operator splitting solver for quadratic programs , 2017, 2018 UKACC 12th International Conference on Control (CONTROL).

[39]  Ernest K. Ryu.  Cosmic divergence, weak cosmic convergence, and fixed points at infinity , 2017, Journal of Fixed Point Theory and Applications.

[40]  Pablo A. Parrilo,et al.  Basis selection for SOS programs via facial reduction and polyhedral approximations , 2014, 53rd IEEE Conference on Decision and Control.

[41]  Damek Davis,et al.  Convergence Rate Analysis of Several Splitting Schemes , 2014, 1406.4834.

[42]  Kim-Chuan Toh,et al.  A note on the convergence of ADMM for linearly constrained convex optimization problems , 2015, Computational Optimization and Applications.

[43]  Wotao Yin,et al.  A new use of Douglas–Rachford splitting for identifying infeasible, unbounded, and pathological conic programs , 2019, Math. Program..

[44]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[45]  Walaa M. Moursi,et al.  The Forward–Backward Algorithm and the Normal Problem , 2016, J. Optim. Theory Appl..

[46]  Heinz H. Bauschke,et al.  Attouch-Théra duality revisited: Paramonotonicity and operator splitting , 2011, J. Approx. Theory.

[47]  Pablo A. Parrilo,et al.  Computation with Polynomial Equations and Inequalities Arising in Combinatorial Optimization , 2009, 0909.0808.

[48]  Matthew K. Tam,et al.  DOUGLAS–RACHFORD FEASIBILITY METHODS FOR MATRIX COMPLETION PROBLEMS , 2013, The ANZIAM Journal.

[49]  Shuzhong Zhang,et al.  Duality Results for Conic Convex Programming , 1997 .

[50]  Z. Luo,et al.  Conic convex programming and self-dual embedding , 1998 .

[51]  Jonathan Eckstein.  Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[52]  R. Rockafellar.  Conjugate Duality and Optimization , 1987 .

[53]  Simon P. Schurr,et al.  Preprocessing and Regularization for Degenerate Semidefinite Programs , 2013 .

[54]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[55]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[56]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[57]  Bolor Jargalsaikhan,et al.  Linear conic programming: genericity and stability , 2015 .

[58]  Henry Wolkowicz,et al.  Strong duality and minimal representations for cone optimization , 2012, Computational Optimization and Applications.

[59]  Stephen P. Boyd,et al.  Infeasibility Detection in the Alternating Direction Method of Multipliers for Convex Optimization , 2019, J. Optim. Theory Appl..

[60]  Gábor Pataki.  A Simple Derivation of a Facial Reduction Algorithm and Extended Dual Systems , .

[61]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[62]  A. Bemporad,et al.  Forward-backward truncated Newton methods for convex composite optimization , 2014, 1402.6655.

[63]  Heinz H. Bauschke,et al.  On the Douglas–Rachford algorithm , 2016, Mathematical Programming.

[64]  Jonathan M. Borwein,et al.  Global behavior of the Douglas–Rachford method for a nonconvex feasibility problem , 2015, J. Glob. Optim..

[65]  H. H. Rachford,et al.  The Numerical Solution of Parabolic and Elliptic Differential Equations , 1955 .

[66]  A. Pazy.  Asymptotic behavior of contractions in hilbert space , 1971 .

[67]  H. Attouch.  A General Duality Principle for the Sum of Two Operators 1 , 1996 .

[68]  Masakazu Muramatsu,et al.  Strange behaviors of interior-point methods for solving semidefinite programming problems in polynomial optimization , 2012, Comput. Optim. Appl..

[69]  Masakazu Muramatsu,et al.  Facial Reduction Algorithms for Conic Optimization Problems , 2012, Journal of Optimization Theory and Applications.

[70]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[71]  Masakazu Muramatsu,et al.  A facial reduction algorithm for finding sparse SOS representations , 2010, Oper. Res. Lett..

[72]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[73]  Stephen P. Boyd,et al.  A Primer on Monotone Operator Methods , 2015 .

[74]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[75]  Jonathan M. Borwein,et al.  The Cyclic Douglas-Rachford Method for Inconsistent Feasibility Problems , 2013, 1310.2195.

[76]  A. Tucker,et al.  Linear Inequalities And Related Systems , 1956 .

[77]  Alberto Bemporad,et al.  Douglas-rachford splitting: Complexity estimates and accelerated variants , 2014, 53rd IEEE Conference on Decision and Control.

[78]  Michel Théra,et al.  Enlargements and Sums of Monotone Operators , 2001 .

[79]  Heinz H. Bauschke,et al.  On the Range of the Douglas-Rachford Operator , 2014, Math. Oper. Res..

[80]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[81]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[82]  Panagiotis Patrinos,et al.  Forward–backward quasi-Newton methods for nonsmooth optimization problems , 2016, Computational Optimization and Applications.

[83]  Hayato Waki,et al.  How to generate weakly infeasible semidefinite programs via Lasserre’s relaxations for polynomial optimization , 2011, Optimization Letters.

[84]  Heinz H. Bauschke,et al.  The Douglas-Rachford Algorithm for Two (Not Necessarily Intersecting) Affine Subspaces , 2015, SIAM J. Optim..

[85]  Masakazu Muramatsu,et al.  A structural geometrical analysis of weakly infeasible SDPs , 2015 .

[86]  J. Borwein,et al.  Facial reduction for a cone-convex programming problem , 1981, Journal of the Australian Mathematical Society. Series A. Pure Mathematics and Statistics.

[87]  Frank Permenter,et al.  Solving Conic Optimization Problems via Self-Dual Embedding and Facial Reduction: A Unified Approach , 2017, SIAM J. Optim..