Foundational Factorization Algorithms for the Efficient Roundoff-Error-Free Solution of Optimization Problems

LU and Cholesky factorizations play a central role in solving linear and mixedinteger programs. In many documented cases, the roundoff errors accrued during the construction and implementation of these factorizations cause the misclassification of suboptimal solutions as optimal and infeasible problems as feasible and viceversa. Such erroneous outputs bring the reliability of optimization solvers into question and, therefore, it is imperative to eliminate these roundoff errors altogether and to do so efficiently to ensure practicality. Firstly, this work introduces two roundoff-error-free factorizations (REF) constructed exclusively in integer arithmetic: the REF LU and Cholesky factorizations. Additionally, it develops supplementary integer-preserving substitution algorithms, thereby providing a complete tool set for solving systems of linear equations (SLEs) exactly and efficiently. An inherent property of the REF factorization algorithms is that their entries’ bit-length—i.e., the number of bits required for expression—is bounded polynomially. Unlike the exact rational arithmetic methods used in practice, however, the algorithms herein presented do not require any greatest common divisor operations to guarantee this pivotal property. Secondly, this work derives various useful theoretical results and details computational tests to demonstrate that the REF factorization framework is considerably superior to the rational arithmetic LU factorization approach in computational performance and storage requirements. This is significant because the latter approach is the solution validation tool of choice of state-of-the-art exact linear programming solvers due to its ability to handle both numerically difficult and intricate problems. An additional theoretical contribution and further computational tests also demon-

[1]  Thorsten Koch The final NETLIB-LP results , 2004, Oper. Res. Lett..

[2]  Arnold Schönhage,et al.  Schnelle Berechnung von Kettenbruchentwicklungen , 1971, Acta Informatica.

[3]  William J. Cook,et al.  Solving Very Sparse Rational Systems of Equations , 2011, TOMS.

[4]  Anthony V. Fiacco,et al.  The Sequential Unconstrained Minimization Technique (SUMT) Without Parameters , 1967, Oper. Res..

[5]  William H. Press,et al.  Numerical Recipes 3rd Edition: The Art of Scientific Computing , 2007 .

[6]  Bernd Gärtner,et al.  Exact arithmetic at low cost—a case study in linear programming , 1999, SODA '98.

[7]  Robert A. van de Geijn,et al.  Updating an LU Factorization with Pivoting , 2008, TOMS.

[8]  John K. Reid,et al.  A sparsity-exploiting variant of the Bartels—Golub decomposition for linear programming bases , 1982, Math. Program..

[9]  E. Berlekamp Factoring polynomials over finite fields , 1967 .

[10]  Daniel E. Steffy,et al.  Konrad-zuse-zentrum F ¨ Ur Informationstechnik Berlin Improving the Accuracy of Linear Programming Solvers with Iterative Refinement Improving the Accuracy of Linear Programming Solvers with Iterative Refinement * , 2022 .

[11]  Douglas H. Wiedemann Solving sparse linear equations over finite fields , 1986, IEEE Trans. Inf. Theory.

[12]  M. Saunders Large-scale linear programming using the Cholesky factorization , 1972 .

[13]  Donald Ervin Knuth,et al.  The Art of Computer Programming, Volume II: Seminumerical Algorithms , 1970 .

[14]  J. Edmonds,et al.  Note sur les $Q$-matrices d'Edmonds , 1997 .

[15]  James L. Massey,et al.  Shift-register synthesis and BCH decoding , 1969, IEEE Trans. Inf. Theory.

[16]  Erich Kaltofen,et al.  Distributed Matrix-Free Solution of Large Sparse Linear Systems over Finite Fields , 1999, Algorithmica.

[17]  Nikolaos V. Sahinidis,et al.  A review of the LU update in the simplex algorithm , 2012, Int. J. Math. Oper. Res..

[18]  S. Barry Cooper,et al.  Rounding-off Errors in Matrix Processes , 2013 .

[19]  B. David Saunders,et al.  Fraction Free Gaussian Elimination for Sparse Matrices , 1995, J. Symb. Comput..

[20]  Lam Lay-Yong,et al.  Methods of solving linear equations in traditional China , 1989 .

[21]  David K. Smith Theory of Linear and Integer Programming , 1987 .

[22]  Ed Klotz,et al.  Identification, Assessment, and Correction of Ill-Conditioning and Numerical Instability in Linear and Integer Programs , 2014 .

[23]  P. Gill,et al.  Maintaining LU factors of a general sparse matrix , 1987 .

[24]  E. Bareiss Sylvester’s identity and multistep integer-preserving Gaussian elimination , 1968 .

[25]  Erich Kaltofen,et al.  On Wiedemann's Method of Solving Sparse Linear Systems , 1991, AAECC.

[26]  George E. Collins,et al.  Efficient Rational Number Reconstruction , 1995, J. Symb. Comput..

[27]  Gilles Villard,et al.  Solving sparse rational linear systems , 2006, ISSAC '06.

[28]  Jean-Guillaume Dumas,et al.  Dense Linear Algebra over Word-Size Prime Fields: the FFLAS and FFPACK Packages , 2006, TOMS.

[29]  Tudor Jebelean,et al.  Practical integer division with Karatsuba complexity , 1997, ISSAC.

[30]  Zhendong Wan,et al.  An algorithm to solve integer linear systems exactly using numerical methods , 2006, J. Symb. Comput..

[31]  V. Klee,et al.  HOW GOOD IS THE SIMPLEX ALGORITHM , 1970 .

[32]  Liang Chen,et al.  Algorithms for solving linear systems over cyclotomic fields , 2010, J. Symb. Comput..

[33]  Arne Storjohann,et al.  Rational solutions of singular linear systems , 2000, ISSAC.

[34]  Silvio Ursic,et al.  Exact Solution of Systems of Linear Equations with Iterative Methods , 1983 .

[35]  Kurt Mehlhorn,et al.  Certifying and repairing solutions to large LPs how good are LP-solvers? , 2003, SODA '03.

[36]  George Havas,et al.  On the worst-case complexity of integer Gaussian elimination , 1997, ISSAC.

[37]  François Margot,et al.  Testing cut generators for mixed-integer linear programming , 2009, Math. Program. Comput..

[38]  David Engel Computer Solution Of Linear Algebraic Systems , 2016 .

[39]  David J. Jeffrey,et al.  Fraction-free matrix factors: new forms for LU and QR factors , 2008, Frontiers of Computer Science in China.

[40]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[41]  J. Stein Computational problems associated with Racah algebra , 1967 .

[42]  William J. Cook,et al.  Topics in exact precision mathematical programming , 2010 .

[43]  Daniel E. Steffy,et al.  Iterative Refinement for Linear Programming , 2016, INFORMS J. Comput..

[44]  Uwe H. Suhl,et al.  A fast LU update for linear programming , 1993, Ann. Oper. Res..

[45]  J. Edmonds Systems of distinct representatives and linear algebra , 1967 .

[46]  D. H. Lehmer Euclid's Algorithm for Large Numbers , 1938 .

[47]  William J. Cook,et al.  Exact solutions to linear programming problems , 2007, Oper. Res. Lett..

[48]  Jean François Pique,et al.  A revised simplex method with integer Q-matrices , 2001, TOMS.

[49]  Ambros M. Gleixner Exact and Fast Algorithms for Mixed-Integer Nonlinear Programming , 2015 .

[50]  Uwe H. Suhl,et al.  Computing Sparse LU Factorizations for Large-Scale Linear Programming Bases , 1990, INFORMS J. Comput..

[51]  Arnold Neumaier,et al.  Safe bounds in linear and mixed-integer linear programming , 2004, Math. Program..

[52]  Jacek Gondzio,et al.  Interior point methods 25 years later , 2012, Eur. J. Oper. Res..

[53]  W. Morven Gentleman,et al.  Analysis of Algorithms, A Case Study: Determinants of Matrices with Polynomial Entries , 1976, TOMS.

[54]  Arnold Schönhage,et al.  Schnelle Multiplikation großer Zahlen , 1971, Computing.

[55]  Erick Moreno-Centeno,et al.  Roundoff-Error-Free Algorithms for Solving Linear Systems via Cholesky and LU Factorizations , 2015, INFORMS J. Comput..

[56]  Roland Wunderling,et al.  Paralleler und objektorientierter Simplex-Algorithmus , 1996 .

[57]  Niels Möller On Schönhage's algorithm and subquadratic integer gcd computation , 2008, Math. Comput..

[58]  H. Wilcox Elementary Linear Algebra , 1970 .

[59]  James Hardy Wilkinson,et al.  Rounding errors in algebraic processes , 1964, IFIP Congress.

[60]  Peter R. Turner,et al.  Fraction-free algorithms for linear and polynomial equations , 1997, SIGS.

[61]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[62]  Gene H. Golub,et al.  The simplex method of linear programming using LU decomposition , 1969, Commun. ACM.

[63]  Arne Storjohann,et al.  A BLAS based C library for exact linear algebra on integer matrices , 2005, ISSAC.

[64]  J. Stillwell Mathematics and Its History , 2020, Undergraduate Texts in Mathematics.

[65]  John Abbott,et al.  How Tight is Hadamard's Bound? , 2001, Exp. Math..

[66]  Joachim von zur Gathen,et al.  Modern Computer Algebra , 1998 .

[67]  Erwin H. Bareiss,et al.  Computational Solutions of Matrix Problems Over an Integral Domain , 1972 .

[68]  Daniel E. Steffy Exact solutions to linear systems of equations using output sensitive lifting , 2011, ACCA.

[69]  Erich Kaltofen An output-sensitive variant of the baby steps/giant steps determinant algorithm , 2002, ISSAC '02.

[70]  Hanif D. Sherali,et al.  Linear Programming and Network Flows , 1977 .

[71]  Numerische Mathematik Exact Solution of Linear Equations Using P-Adie Expansions* , 2005 .

[72]  John A. Tomlin,et al.  Updated triangular factors of the basis to maintain sparsity in the product form simplex method , 1972, Math. Program..

[73]  Daniel G. Espinoza On Linear Programming, Integer Programming and Cutting Planes , 2006 .