Operator splitting for a homogeneous embedding of the monotone linear complementarity problem

In this work we apply Douglas-Rachford splitting to a homogeneous embedding of the linear complementarity problem (LCP). The homogeneous embedding converts LCPs into non-linear monotone complementarity problems (MCPs) that encode both the optimality conditions of the original LCP as well as certificates of infeasibility. The resulting problem can be expressed as the problem of finding a zero of the sum of two maximal monotone operators, to which we apply operator splitting. The resulting algorithm has almost identical per-iteration cost as applying Douglas-Rachford splitting to the LCP directly. Specifically it requires solving a linear system and projecting onto a cone at every iteration. Previous work applied operator splitting to a homogeneous embedding of a linear cone optimization problem. This paper extends the analysis to cover quadratic objectives, due to the correspondence between quadratic programming and the linear complementarity problem. Our algorithm is able to return a primal-dual solution to the quadratic cone program should one exist, or a certificate of infeasibility otherwise. We provide extensive numerical experimentation to show that for feasible problems our approach tends to be somewhat faster than applying operator splitting directly to the LCP, and in cases of infeasibility our approach can be orders of magnitude faster than competing approaches based on differences of iterates.

[1]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[2]  Yinyu Ye,et al.  On a homogeneous algorithm for the monotone complementarity problem , 1999, Math. Program..

[3]  Stephen P. Boyd,et al.  OSQP: an operator splitting solver for quadratic programs , 2017, 2018 UKACC 12th International Conference on Control (CONTROL).

[4]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[5]  Mark Cannon,et al.  COSMO: A conic operator splitting method for large convex problems , 2019, 2019 18th European Control Conference (ECC).

[6]  Wotao Yin,et al.  A New Use of Douglas-Rachford Splitting and ADMM for Identifying Infeasible, Unbounded, and Pathological Conic Programs , 2017, ArXiv.

[7]  Pantelis Sopasakis,et al.  SuperSCS: fast and accurate large-scale conic optimization , 2019, 2019 18th European Control Conference (ECC).

[8]  Robert J. Vanderbei,et al.  Symmetric Quasidefinite Matrices , 1995, SIAM J. Optim..

[9]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[10]  Katta G. Murty,et al.  Linear complementarity, linear and nonlinear programming , 1988 .

[11]  Shinji Mizuno,et al.  An O(√nL)-Iteration Homogeneous and Self-Dual Linear Programming Algorithm , 1994, Math. Oper. Res..

[12]  Elmer Earl. Branstetter,et al.  The theory of linear programming , 1963 .

[13]  G. Minty On the maximal domain of a ``monotone'' function. , 1961 .

[14]  Brendan O'Donoghue,et al.  Hamiltonian descent for composite objectives , 2019, NeurIPS.

[15]  Richard W. Cottle,et al.  Linear Complementarity Problem , 2009, Encyclopedia of Optimization.

[16]  G. Minty Monotone (nonlinear) operators in Hilbert space , 1962 .

[17]  D. Gabay Applications of the method of multipliers to variational inequalities , 1983 .

[18]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[19]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[20]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[21]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[22]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[23]  Timothy A. Davis,et al.  Direct methods for sparse linear systems , 2006, Fundamentals of algorithms.

[24]  Panagiotis Patrinos,et al.  SuperMann: A Superlinearly Convergent Algorithm for Finding Fixed Points of Nonexpansive Operators , 2016, IEEE Transactions on Automatic Control.

[25]  Richard W. Cottle,et al.  Linear Complementarity Problem. , 1992 .

[26]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[27]  Stephen P. Boyd,et al.  Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding , 2013, Journal of Optimization Theory and Applications.

[28]  Yang Zheng,et al.  Chordal decomposition in operator-splitting methods for sparse semidefinite programs , 2017, Mathematical Programming.

[29]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[30]  Frank Permenter,et al.  Solving Conic Optimization Problems via Self-Dual Embedding and Facial Reduction: A Unified Approach , 2017, SIAM J. Optim..

[31]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[32]  Stephen P. Boyd,et al.  A Splitting Method for Optimal Control , 2013, IEEE Transactions on Control Systems Technology.

[33]  Yinyu Ye,et al.  On homogeneous and self-dual algorithms for LCP , 1997, Math. Program..

[34]  Lieven Vandenberghe,et al.  Douglas–Rachford Splitting for the Sum of a Lipschitz Continuous and a Strongly Monotone Operator , 2019, J. Optim. Theory Appl..

[35]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[36]  A. Nemirovski Advances in convex optimization : conic programming , 2005 .

[37]  Stephen P. Boyd,et al.  Infeasibility Detection in the Alternating Direction Method of Multipliers for Convex Optimization , 2018, Journal of Optimization Theory and Applications.

[38]  Richard G. Baraniuk,et al.  Fast Alternating Direction Optimization Methods , 2014, SIAM J. Imaging Sci..

[39]  Stephen P. Boyd,et al.  A Primer on Monotone Operator Methods , 2015 .

[40]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[41]  H. H. Rachford,et al.  The Numerical Solution of Parabolic and Elliptic Differential Equations , 1955 .

[42]  Stephen P. Boyd,et al.  Globally Convergent Type-I Anderson Acceleration for Nonsmooth Fixed-Point Iterations , 2018, SIAM J. Optim..