Limited-memory LDL⊤ factorization of symmetric quasi-definite matrices with application to constrained optimization

We propose a generalization of the limited-memory Cholesky factorization of Lin and Moré (SIAM J. Sci. Comput. 21(1), 24–45, 1999) to the symmetric indefinite case with special interest in symmetric quasi-definite matrices. We use this incomplete factorization to precondition two formulations of linear systems arising from regularized interior-point methods for quadratic optimization. An advantage of the limited-memory approach is predictable memory requirements. We establish existence of incomplete factors when the input matrix is an H-matrix but our numerical results illustrate that the factorization succeeds more generally. An appropriate diagonal shift is applied whenever the input matrix is not quasi definite. As the memory parameter increases an efficiency measure of the preconditioner suggested by Scott and Tůma (2013) improves. The combination of the 3×3 block formulation analyzed by Greif, Moulding, and Orban (SIAM J. Optim. 24(1), 49–83, 2014), the SYMAMD ordering, and a moderate memory parameter results in encouraging performance.

[1]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[2]  Mark T. Jones,et al.  An improved incomplete Cholesky factorization , 1995, TOMS.

[3]  E. Cuthill,et al.  Reducing the bandwidth of sparse symmetric matrices , 1969, ACM '69.

[4]  Barry Lee,et al.  Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics , 2006, Math. Comput..

[5]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[6]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[7]  Owe Axelsson,et al.  Preconditioning methods for linear systems arising in constrained optimization problems , 2003, Numer. Linear Algebra Appl..

[8]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[9]  Patrick R. Amestoy,et al.  An Approximate Minimum Degree Ordering Algorithm , 1996, SIAM J. Matrix Anal. Appl..

[10]  Howard C. Elman,et al.  Finite Elements and Fast Iterative Solvers: with Applications in Incompressible Fluid Dynamics , 2014 .

[11]  C. Mészáros,et al.  A repository of convex quadratic programming problems , 1999 .

[12]  J. Burke,et al.  Optimization viewpoint on Kalman smoothing, with applications to robust and sparse estimation , 2013, 1303.1993.

[13]  Alan George,et al.  A linear time implementation of the reverse Cuthill-McKee algorithm , 1980, BIT.

[14]  John J. Buoni A stable method for the incomplete factorization of H-matrices , 1990 .

[15]  J. Ortega Introduction to Parallel and Vector Solution of Linear Systems , 1988, Frontiers of Computer Science.

[16]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[17]  Dominique Orban,et al.  Bounds on Eigenvalues of Matrices Arising from Interior-Point Methods , 2012, SIAM J. Optim..

[18]  Nicholas I. M. Gould,et al.  Implicit-Factorization Preconditioning and Iterative Solvers for Regularized Saddle-Point Systems , 2006, SIAM J. Matrix Anal. Appl..

[19]  Philippe Courtier,et al.  Dual formulation of four‐dimensional variational assimilation , 1997 .

[20]  Na Li,et al.  Crout versions of ILU factorization with pivoting for sparse symmetric matrices. , 2005 .

[21]  Nicholas I. M. Gould,et al.  Numerical methods for large-scale nonlinear optimization , 2005, Acta Numerica.

[22]  Jorge Nocedal,et al.  Knitro: An Integrated Package for Nonlinear Optimization , 2006 .

[23]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[24]  Robert J. Vanderbei,et al.  An Interior-Point Algorithm for Nonconvex Nonlinear Programming , 1999, Comput. Optim. Appl..

[25]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[26]  M. Tismenetsky,et al.  A new preconditioning technique for solving large sparse linear systems , 1991 .

[27]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[28]  Chih-Jen Lin,et al.  Incomplete Cholesky Factorizations with Limited Memory , 1999, SIAM J. Sci. Comput..

[29]  Dominique Orban,et al.  Iterative Methods for Symmetric Quasi-Definite Linear Systems. Part I: Theory , 2013 .

[30]  A. Messaoudi,et al.  On the stability of the incomplete LU-factorizations and characterizations of H-matrices , 1995 .

[31]  I. Gustafsson A class of first order factorization methods , 1978 .

[32]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[33]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[34]  R. Vanderbei Symmetric Quasi-Definite Matrices , 2006 .

[35]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[36]  Michael P. Friedlander,et al.  A primal–dual regularized interior-point method for convex quadratic programs , 2010, Mathematical Programming Computation.

[37]  T. Manteuffel An incomplete factorization technique for positive definite linear systems , 1980 .

[38]  Alan George,et al.  On the Condition of Symmetric Quasi-Definite Matrices , 2000, SIAM J. Matrix Anal. Appl..

[39]  Jennifer A. Scott,et al.  HSL_MI28 , 2014, ACM Trans. Math. Softw..

[40]  N. Munksgaard,et al.  Solving Sparse Symmetric Sets of Linear Equations by Preconditioned Conjugate Gradients , 1980, TOMS.

[41]  Joseph R. Shinnerl,et al.  On the Stability of Cholesky Factorization for Symmetric Quasidefinite Systems , 1996, SIAM J. Matrix Anal. Appl..

[42]  O. Axelsson Iterative solution methods , 1995 .

[43]  Miroslav Tuma,et al.  A Note on the LDLT Decomposition of Matrices from Saddle-Point Problems , 2001, SIAM J. Matrix Anal. Appl..

[44]  PDCO : Primal-Dual Interior Methods 1 Interior methods for linear optimization , .

[45]  Yousef Saad,et al.  ILUT: A dual threshold incomplete LU factorization , 1994, Numer. Linear Algebra Appl..

[46]  Edmond Chow,et al.  Crout Versions of ILU for General Sparse Matrices , 2003, SIAM J. Sci. Comput..

[47]  Robert J. Vanderbei,et al.  Symmetric Quasidefinite Matrices , 1995, SIAM J. Optim..

[48]  Igor E. Kaporin,et al.  High quality preconditioning of a general symmetric positive definite matrix based on its U , 1998 .

[49]  Gene H. Golub,et al.  Matrix computations , 1983 .

[50]  Igor E. Kaporin,et al.  High quality preconditioning of a general symmetric positive definite matrix based on its UTU + UTR + RTU-decomposition , 1998, Numer. Linear Algebra Appl..