Implementation of interior-point methods for LP based on Krylov subspace iterative solvers with inner-iteration preconditioning

We apply novel inner-iteration preconditioned Krylov subspace methods to the interior-point algorithm for linear programming (LP). Inner-iteration preconditioners recently proposed by Morikuni and Hayami enable us to overcome the severe ill-conditioning of linear equations solved in the final phase of interior-point iterations. The Krylov subspace methods do not suffer from rank-deficiency and therefore no preprocessing is necessary even if rows of the constraint matrix are not linearly independent. By means of these methods, a new interior-point recurrence is proposed in order to omit one matrix-vector product at each step. Extensive numerical experiments are conducted over diverse instances of 140 LP problems including the Netlib, QAPLIB, Mittelmann and Atomizer Basis Pursuit collections. The largest problem has 434,580 unknowns. It turns out that our implementation is more robust than the standard public domain solvers SeDuMi (Self-Dual Minimization), SDPT3 (Semidefinite Programming Toh-Todd-Tütüncü) and the LSMR iterative solver in PDCO (Primal-Dual Barrier Method for Convex Objectives) without increasing CPU time. The proposed interior-point method based on iterative solvers succeeds in solving a fairly large number of LP instances from benchmark libraries under the standard stopping criteria. The work also presents a fairly extensive benchmark test for several renowned solvers including direct and iterative solvers.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[3]  E. J. Craig.  The N‐Step Iteration Procedures , 1955 .

[4]  M. Sakarovitch,et al.  Two commodity network flows and linear programming , 1973, Math. Program..

[5]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[6]  Å. Björck,et al.  Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations , 1979 .

[7]  Michael A. Saunders,et al.  On projected newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method , 1986, Math. Program..

[8]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[9]  Kunio Tanabe,et al.  Centered newton method for mathematical programming , 1988 .

[10]  R. C. Monteiro,et al.  Interior path following primal-dual algorithms , 1988 .

[11]  Shinji Mizuno,et al.  A polynomial-time algorithm for a class of linear complementarity problems , 1989, Math. Program..

[12]  Mauricio G. C. Resende,et al.  An implementation of Karmarkar's algorithm for linear programming , 1989, Math. Program..

[13]  Renato D. C. Monteiro,et al.  Interior path following primal-dual algorithms. part I: Linear programming , 1989, Math. Program..

[14]  Achiya Dax,et al.  The Convergence of Linear Stationary Iterative Processes for Solving Singular Unstructured Systems of Linear Equations , 1990, SIAM Rev..

[15]  K. G. Ramakrishnan,et al.  Computational results of an interior point algorithm for large scale linear programming , 1991, Math. Program..

[16]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[17]  Roy E. Marsten,et al.  On Implementing Mehrotra's Predictor-Corrector Interior-Point Method for Linear Programming , 1992, SIAM J. Optim..

[18]  Sanjay Mehrotra,et al.  Implementations of Affine Scaling Methods: Approximate Solutions of Systems of Linear Equations Using Preconditioned Conjugate Gradient Methods , 1992, INFORMS J. Comput..

[19]  Sanjay Mehrotra,et al.  Solving symmetric indefinite systems in an interior-point method for linear programming , 1993, Math. Program..

[20]  M. Resende,et al.  An Implementation of the Dual Affine Scaling Algorithm for Minimum-Cost Flow on Bipartite Uncapacitated Networks , 1993, SIAM J. Optim..

[21]  David F. Shanno,et al.  An interior point method for quadratic programs based on conjugate projected gradients , 1993, Comput. Optim. Appl..

[22]  A. Vannelli,et al.  PCG techniques for interior point algorithms , 1993, Proceedings of 36th Midwest Symposium on Circuits and Systems.

[23]  Roy E. Marsten,et al.  Feature Article - Interior Point Methods for Linear Programming: Computational State of the Art , 1994, INFORMS J. Comput..

[24]  Yin Zhang,et al.  On the Convergence of a Class of Infeasible Interior-Point Methods for the Horizontal Linear Complementarity Problem , 1994, SIAM J. Optim..

[25]  Tsuneaki Miyahara,et al.  Graduate university of advanced studies , 1994 .

[26]  Erling D. Andersen,et al.  Presolving in linear programming , 1995, Math. Program..

[27]  J. Gondzio.  HOPDM (version 2.12) — A fast LP solver based on a primal-dual interior point method , 1995 .

[28]  Jacek Gondzio,et al.  Multiple centrality corrections in a primal-dual method for linear programming , 1996, Comput. Optim. Appl..

[29]  Jacek Gondzio,et al.  Implementation of Interior Point Methods for Large Scale Linear Programming , 1996 .

[30]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[31]  Tamás Terlaky,et al.  A computational view of interior point methods , 1996 .

[32]  Roland W. Freund,et al.  A QMR-based interior-point algorithm for solving linear programs , 1997, Math. Program..

[33]  J. L. Nazareth,et al.  Linear and nonlinear conjugate gradient-related methods , 1996 .

[34]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[35]  Jacek Gondzio,et al.  Presolove Analysis of Linear Programs Prior to Applying an Interior Point Method , 1997, INFORMS J. Comput..

[36]  T Talaky,et al.  Interior Point Methods of Mathematical Programming , 1997 .

[37]  Yin Zhang,et al.  Solving large-scale linear programs by interior-point methods under the Matlab ∗ Environment † , 1998 .

[38]  Stephen J. Wright.  Modified Cholesky Factorizations in Interior-Point Algorithms for Linear Programming , 1999, SIAM J. Optim..

[39]  Shinji Mizuno,et al.  Convergence of a Class of Inexact Interior-Point Algorithms for Linear Programs , 1999, Math. Oper. Res..

[40]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[41]  Stephen J. Wright,et al.  PCx: an interior-point code for linear programming , 1999 .

[42]  L. Portugal,et al.  A truncated primal‐infeasible dual‐feasible network interior point method , 2000 .

[43]  Janos Korzak,et al.  Convergence Analysis of Inexact Infeasible-Interior-Point Algorithms for Solving Linear Programming Problems , 2000, SIAM J. Optim..

[44]  L. Portugal,et al.  A truncated primal-infeasible dual-feasible network interior point method , 2000, Networks.

[45]  D. Donoho,et al.  Atomic Decomposition by Basis Pursuit , 2001 .

[46]  Andrew G. Glen,et al.  APPL , 2001 .

[47]  Michael C. Ferris,et al.  Interior-Point Methods for Massive Support Vector Machines , 2002, SIAM J. Optim..

[48]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[49]  Joaquim Júdice,et al.  A Study of Preconditioners for Network Interior Point Methods , 2003, Comput. Optim. Appl..

[50]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[51]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[52]  Jerome W. O'Neal,et al.  Convergence Analysis of a Long-Step Primal-Dual Infeasible Interior-Point LP Algorithm Based on Iterative Linear Solvers , 2003 .

[53]  Takashi Tsuchiya,et al.  Uniform Boundedness of a Preconditioned Normal Matrix Used in Interior-Point Methods , 2004, SIAM J. Optim..

[54]  Luca Bergamaschi,et al.  Preconditioning Indefinite Systems in Interior Point Methods for Optimization , 2004, Comput. Optim. Appl..

[55]  Dianne P. O'Leary,et al.  Adaptive use of iterative methods in predictor–corrector interior point methods for linear programming , 2000, Numerical Algorithms.

[56]  Iain S. Duff,et al.  MA57---a code for the solution of sparse symmetric definite and indefinite systems , 2004, TOMS.

[57]  D. Sorensen,et al.  A new class of preconditioners for large-scale linear systems from interior point methods for linear programming , 2005 .

[58]  Sanjay Mehrotra,et al.  Convergence Conditions and Krylov Subspace--Based Corrections for Primal-Dual Interior-Point Method , 2005, SIAM J. Optim..

[59]  Luca Bergamaschi,et al.  Erratum to: Inexact constraint preconditioners for linear systems arising in interior point methods , 2011, Comput. Optim. Appl..

[60]  Chen Greif,et al.  A Preconditioner for Linear Systems Arising From Interior Point Optimization Methods , 2007, SIAM J. Sci. Comput..

[61]  Jacek Gondzio,et al.  Preconditioning indefinite systems in interior point methods for large scale linear optimisation , 2008, Optim. Methods Softw..

[62]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[63]  Xiaoke Cui.  Approximate Generalized Inverse Preconditioning Methods for Least Squares Problems , 2009 .

[64]  J. Gondzio,et al.  Convergence Analysis of the Inexact Infeasible Interior-Point Method for Linear Optimization , 2009 .

[65]  Jun-Feng Yin,et al.  GMRES Methods for Least Squares Problems , 2010, SIAM J. Matrix Anal. Appl..

[66]  Daniela di Serafino,et al.  DIPARTIMENTO DI MATEMATICA , 2008 .

[67]  Jun-Feng Yin,et al.  Greville’s method for preconditioning least squares problems , 2011, Adv. Comput. Math..

[68]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[69]  Jacek Gondzio,et al.  Matrix-free interior point method , 2012, Comput. Optim. Appl..

[70]  Jacek Gondzio,et al.  Interior point methods 25 years later , 2012, Eur. J. Oper. Res..

[71]  Keiichi Morikuni,et al.  Inner-Iteration Krylov Subspace Methods for Least Squares Problems , 2013, SIAM J. Matrix Anal. Appl..

[72]  Keiichi Morikuni,et al.  Convergence of Inner-Iteration GMRES Methods for Rank-Deficient Least Squares Problems , 2015, SIAM J. Matrix Anal. Appl..

[73]  Keiichi Morikuni,et al.  Symmetric inner-iteration preconditioning for rank-deficient least squares problems , 2015, 1504.00889.