A Primal-Dual Operator Splitting Method for Conic Optimization

We develop a simple operator splitting method for solving a primal conic optimization problem; we show that the iterates also solve the dual problem. The resulting algorithm is very simple to describe and implement and yields solutions of modest accuracy in competitive times. Several versions of the algorithm are amenable to parallelization, either via distributed linear algebra or GPU-accelerated matrix-vector multiplication. We provide a simple, single-threaded C implementation for reference. Electrical Engineering Department, Stanford University. Email: {echu508, boyd}@stanford.edu Quantcast. Email: bodonoghue85@gmail.com Computer Science Department, Stanford University. Email: npparikh@cs.stanford.edu

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  E. E. Osborne.  On Pre-Conditioning of Matrices , 1960, JACM.

[3]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[4]  Gene H. Golub,et al.  Matrix computations , 1983 .

[5]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[6]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[7]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[8]  Jonathan Eckstein.  Parallel alternating direction multiplier decomposition of convex programs , 1994 .

[9]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[10]  Shinji Mizuno,et al.  An O(√nL)-Iteration Homogeneous and Self-Dual Linear Programming Algorithm , 1994, Math. Oper. Res..

[11]  Joseph R. Shinnerl,et al.  On the Stability of Cholesky Factorization for Symmetric Quasidefinite Systems , 1996, SIAM J. Matrix Anal. Appl..

[12]  Yinyu Ye,et al.  A simplified homogeneous and self-dual linear programming algorithm and its implementation , 1996, Ann. Oper. Res..

[13]  Patrick R. Amestoy,et al.  An Approximate Minimum Degree Ordering Algorithm , 1996, SIAM J. Matrix Anal. Appl..

[14]  J. L. Nazareth,et al.  Cholesky-based Methods for Sparse Least Squares : The Benefits of Regularization ∗ , 1996 .

[15]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[16]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[17]  Michael C. Ferris,et al.  Operator-Splitting Methods for Monotone Affine Variational Inequalities, with a Parallel Application to Optimal Control , 1998, INFORMS J. Comput..

[18]  Stephen P. Boyd,et al.  Applications of second-order cone programming , 1998 .

[19]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[20]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[21]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[22]  D. Ruiz.  A Scaling Algorithm to Equilibrate Both Rows and Columns Norms in Matrices 1 , 2001 .

[23]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[24]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[25]  Timothy A. Davis,et al.  Algorithm 837: AMD, an approximate minimum degree ordering algorithm , 2004, TOMS.

[26]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[27]  Olaf Schenk,et al.  Solving unsymmetric sparse systems of linear equations with PARDISO , 2004, Future Gener. Comput. Syst..

[28]  Timothy A. Davis,et al.  Algorithm 8 xx : a concise sparse Cholesky factorization package , 2004 .

[29]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[30]  O. Schenk,et al.  ON FAST FACTORIZATION PIVOTING METHODS FOR SPARSE SYMMETRI C INDEFINITE SYSTEMS , 2006 .

[31]  Timothy A. Davis,et al.  Summary of available software for sparse direct methods , 2006 .

[32]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[33]  Timothy A. Davis,et al.  Direct methods for sparse linear systems , 2006, Fundamentals of algorithms.

[34]  Olaf Schenk,et al.  Matching-based preprocessing algorithms to the solution of saddle-point problems in large-scale nonconvex interior-point optimization , 2007, Comput. Optim. Appl..

[35]  Kevin Skadron,et al.  Scalable parallel programming , 2008, 2008 IEEE Hot Chips 20 Symposium (HCS).

[36]  Rudolf A. Römer,et al.  On large‐scale diagonalization techniques for the Anderson model of localization , 2005, SIAM J. Sci. Comput..

[37]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[38]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[39]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[40]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[41]  Guanghui Lan,et al.  Primal-dual first-order methods with O (1/e) iteration-complexity for cone programming. , 2011 .

[42]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[43]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[44]  Stephen P. Boyd,et al.  CVXGEN: a code generator for embedded convex optimization , 2011, Optimization and Engineering.

[45]  Stephen P. Boyd,et al.  ECOS: An SOCP solver for embedded systems , 2013, 2013 European Control Conference (ECC).

[46]  N. S. Aybat,et al.  An Augmented Lagrangian Method for Conic Convex Programming , 2013, 1302.6322.

[47]  Jacek Gondzio,et al.  Convergence Analysis of an Inexact Feasible Interior Point Method for Convex Quadratic Programming , 2012, SIAM J. Optim..

[48]  Robert A. van de Geijn,et al.  Elemental: A New Framework for Distributed Memory Dense Matrix Computations , 2013, TOMS.

[49]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[50]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[51]  Jacek Gondzio,et al.  Matrix-free interior point method for compressed sensing problems , 2012, Mathematical Programming Computation.

[52]  Euhanna Ghadimi,et al.  Optimal Parameter Selection for the Alternating Direction Method of Multipliers (ADMM): Quadratic Problems , 2013, IEEE Transactions on Automatic Control.