Modified Cholesky algorithms: a catalog with new approaches

Given an n ×  n symmetric possibly indefinite matrix A, a modified Cholesky algorithm computes a factorization of the positive definite matrix A +  E, where E is a correction matrix. Since the factorization is often used to compute a Newton-like downhill search direction for an optimization problem, the goals are to compute the modification without much additional cost and to keep A +  E well-conditioned and close to A. Gill, Murray and Wright introduced a stable algorithm, with a bound of ||E||2 =  O(n2). An algorithm of Schnabel and Eskow further guarantees ||E||2 =  O(n). We present variants that also ensure ||E||2 =  O(n). Moré and Sorensen and Cheng and Higham used the block LBLT factorization with blocks of order 1 or 2. Algorithms in this class have a worst-case cost O(n3) higher than the standard Cholesky factorization. We present a new approach using a sandwiched LTLT-LBLT factorization, with T tridiagonal, that guarantees a modification cost of at most O(n2).

[1]  G GrimesRoger,et al.  Accurate Symmetric Indefinite Linear Equation Solvers , 1999 .

[2]  J. Cuppen.  A divide and conquer method for the symmetric tridiagonal eigenproblem , 1980 .

[3]  Philip E. Gill,et al.  Practical optimization , 1981 .

[4]  Dianne P. O'Leary,et al.  Stable Factorizations of Symmetric Tridiagonal and Triadic Matrices , 2006, SIAM J. Matrix Anal. Appl..

[5]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[6]  Philip E. Gill,et al.  Newton-type methods for unconstrained and linearly constrained optimization , 1974, Math. Program..

[7]  G. Miller.  On the Solution of a System of Linear Equations , 1910 .

[8]  John G. Lewis,et al.  Accurate Symmetric Indefinite Linear Equation Solvers , 1999, SIAM J. Matrix Anal. Appl..

[9]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[10]  J. Bunch,et al.  Direct Methods for Solving Symmetric Indefinite Systems of Linear Equations , 1971 .

[11]  Danny C. Sorensen,et al.  On the use of directions of negative curvature in a modified newton method , 1979, Math. Program..

[12]  S. H. Cheng,et al.  A Modified Cholesky Algorithm Based on a Symmetric Indefinite Factorization , 1998, SIAM J. Matrix Anal. Appl..

[13]  J. O. Aasen.  On the reduction of a symmetric matrix to tridiagonal form , 1971 .

[14]  Anders Forsgren,et al.  Computing Modified Newton Directions Using a Partial Cholesky Factorization , 1995, SIAM J. Sci. Comput..

[15]  Elizabeth Eskow,et al.  A Revised Modified Cholesky Factorization Algorithm , 1999, SIAM J. Optim..

[16]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[17]  N. Higham.  Stability of the Diagonal Pivoting Method with Partial Pivoting , 1997, SIAM J. Matrix Anal. Appl..

[18]  H. A. Schulke.  Matrix factorization , 1955, IRE Transactions on Circuit Theory.

[19]  G. Stewart.  The Efficient Generation of Random Orthogonal Matrices with an Application to Condition Estimators , 1980 .

[20]  Elizabeth Eskow,et al.  A New Modified Cholesky Factorization , 1990, SIAM J. Sci. Comput..

[21]  Dianne P. O'Leary,et al.  Matrix factorizations, triadic matrices, and modified cholesky factorizations for optimization , 2006 .