We study the use of black-box LDL factorizations for solving the augmented systems (KKT systems) associated with least-squares problems and barrier methods for linear programming (LP). With judicious regularization parameters, stability can be achieved for arbitrary data and arbitrary permutations of the KKT matrix. This offers improved efficiency compared to implementations based on “pure normal equations” or “pure KKT systems”. In particular, the LP matrix may be partitioned arbitrarily as (As Ad ). If AsA T s is unusually sparse, the associated “reduced KKT system” may have very sparse Cholesky factors. Similarly for least-squares problems if a large number of rows of the observation matrix have special structure. Numerical behavior is illustrated on the villainous Netlib models greenbea and pilots. 1 Background The connection between this work and Conjugate-Gradient methods lies in some properties of two CG algorithms, LSQR and CRAIG, for solving linear equations and least-squares problems of various forms. We consider the following problems: Linear equations: Ax = b (1) Minimum length: min ‖x‖ subject to Ax = b (2) Least squares: min ‖Ax− b‖ (3) Regularized least squares: min ‖Ax− b‖ + ‖δx‖ (4) Regularized min length: min ‖x‖ + ‖s‖ subject to Ax+ δs = b (5) where A is a general matrix (square or rectangular) and δ is a scalar (δ ≥ 0). LSQR [17, 18] solves the first four problems, and incidentally the fifth, using essentially the same work and storage per iteration in all cases. The iterates xk reduce ‖b − Axk‖ monotonically. CRAIG [4, 17] solves only compatible systems (1)–(2), with ‖x − xk‖ decreasing monotonically. Since CRAIG is slightly simpler and more economical than LSQR, it may sometimes be preferred for those problems. Partially supported by Department of Energy grant DE-FG03-92ER25117, National Science Foundation grant DMI-9204208, and Office of Naval Research grant N00014-90-J-1242. Systems Optimization Laboratory, Dept of Operations Research, Stanford University, Stanford, California 94305-4022 (mike@SOL-michael.stanford.edu). Cholesky-based Methods for Least Squares 93 To extend CRAIG to incompatible systems, we have studied problem (5): a compatible system in the combined variables (x, s). If δ > 0, it is readily confirmed that problems (4) and (5) have the same solution x, and that both are solved by either the normal equations Nx = Ab, N ≡ AA+ δI, (6) or the augmented system
[1]
E. J. Craig.
The N‐Step Iteration Procedures
,
1955
.
[2]
J. Miller.
Numerical Analysis
,
1966,
Nature.
[3]
G. Golub,et al.
Unsymmetric positive definite linear systems
,
1979
.
[4]
Michael A. Saunders,et al.
LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
,
1982,
TOMS.
[5]
Michael A. Saunders,et al.
Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems
,
1982,
TOMS.
[6]
John K. Reid,et al.
The Multifrontal Solution of Indefinite Sparse Symmetric Linear
,
1983,
TOMS.
[7]
P. Gill,et al.
Solving Reduced KKT Systems in Barrier Methods for Linear and Quadratic Programming
,
1991
.
[8]
Roy E. Marsten,et al.
On Implementing Mehrotra's Predictor-Corrector Interior-Point Method for Linear Programming
,
1992,
SIAM J. Optim..
[9]
Sanjay Mehrotra,et al.
Solving symmetric indefinite systems in an interior-point method for linear programming
,
1993,
Math. Program..
[10]
Robert E. Bixby,et al.
Progress in Linear Programming
,
1993
.
[11]
Roy E. Marsten,et al.
Feature Article - Interior Point Methods for Linear Programming: Computational State of the Art
,
1994,
INFORMS J. Comput..
[12]
S. Vavasis.
Stable Numerical Algorithms for Equilibrium Systems
,
1994,
SIAM J. Matrix Anal. Appl..
[13]
M. Saunders.
Solution of sparse rectangular systems using LSQR and CRAIG
,
1995
.
[14]
Irvin J. Lustig,et al.
Gigaflops in linear programming
,
1996,
Oper. Res. Lett..
[15]
John K. Reid,et al.
Exploiting zeros on the diagonal in the direct solution of indefinite sparse symmetric linear systems
,
1996,
TOMS.
[16]
Anders Forsgren.
On Linear Least-Squares Problems with Diagonally Dominant Weight Matrices
,
1996,
SIAM J. Matrix Anal. Appl..