Linear Programming and Condition Numbers under the Real Number Computation Model

Publisher Summary Linear programming plays a distinguished role in complexity theory. In one sense, it is a continuous optimization problem, because the goal is to minimize a linear objective function over a convex polyhedron. However, it is also a combinatorial problem involving selecting an extreme point among a finite set of possible vertices. Interior-point algorithms are continuous iterative algorithms. Computational experience with sophisticated procedures suggests that the number of necessary iterations grows very slowly with the problem size. This provides the potential for dramatic improvements in computation effectiveness. This chapter describes the complexity theoretic properties of interior-point algorithms. Complexity theory is arguably the foundational stone of computer algorithms. The goal of the theory is twofold: to develop criteria for measuring the effectiveness of various algorithms and to assess the inherent difficulty of various problems. The term “complexity” refers to the amount of resources required by a computation. The chapter discusses a particular resource that is the computing time. In complexity theory, however, the execution time of a program implemented in a particular programming language, running on a particular computer over a particular input is not of interest.

[1]  E. Gilbert.  The probability of covering a sphere with N circular caps , 1965 .

[2]  J. Renegar.  Some perturbation theory for linear programming , 1994, Math. Program..

[3]  R. E. Miles.  The asymptotic values of certain coverage probabilities , 1969 .

[4]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[5]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[6]  J. Demmel.  On condition numbers and the distance to the nearest ill-posed problem , 2015 .

[7]  S. Smale,et al.  On a theory of computation and complexity over the real numbers; np-completeness , 1989 .

[8]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[9]  Jörg M. Wills,et al.  Handbook of Convex Geometry , 1993 .

[10]  B. Ripley,et al.  Introduction to the Theory of Coverage Processes. , 1989 .

[11]  Yinyu Ye,et al.  On the finite convergence of interior-point algorithms for linear programming , 1992, Math. Program..

[12]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[13]  L. Lovász,et al.  Geometric Algorithms and Combinatorial Optimization , 1981 .

[14]  Stephen Smale,et al.  On the average number of steps of the simplex method of linear programming , 1983, Math. Program..

[15]  Michael J. Todd,et al.  Feature Article - The Ellipsoid Method: A Survey , 1981, Oper. Res..

[16]  Robert M. Freund,et al.  Condition-Based Complexity of Convex Optimization in Conic Linear Form via the Ellipsoid Algorithm , 1999, SIAM J. Optim..

[17]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .

[18]  M. Teboulle,et al.  A geometric property of the least squares solution of linear equations , 1990 .

[19]  Felipe Cucker,et al.  A Primal-Dual Algorithm for Solving Polyhedral Conic Systems with a Finite-Precision Machine , 2002, SIAM J. Optim..

[20]  A. Turing.  ROUNDING-OFF ERRORS IN MATRIX PROCESSES , 1948 .

[21]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[22]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[23]  Éva Tardos,et al.  A Strongly Polynomial Algorithm to Solve Combinatorial Linear Programs , 1986, Oper. Res..

[24]  Michael J. Todd,et al.  A Dantzig-Wolfe-Like Variant of Karmarkar's Interior-Point Linear Programming Algorithm , 1990, Oper. Res..

[25]  G. Stewart.  On scaled projections and pseudoinverses , 1989 .

[26]  James Renegar,et al.  Linear programming, complexity theory and elementary functional analysis , 1995, Math. Program..

[27]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[28]  Lenore Blum,et al.  Complexity and Real Computation , 1997, Springer New York.

[29]  J. Hartmanis,et al.  On the Computational Complexity of Algorithms , 1965 .

[30]  Herman Heine Goldstine,et al.  A History of Numerical Analysis from the 16th through the 19th Century. , 1976 .

[31]  James Renegar,et al.  Incorporating Condition Measures into the Complexity Theory of Linear Programming , 1995, SIAM J. Optim..

[32]  Felipe Cucker,et al.  A new condition number for linear programming , 2001, Math. Program..

[33]  Raymond E. Miller,et al.  Complexity of Computer Computations , 1972 .

[34]  Boris A. Trakhtenbrot,et al.  A Survey of Russian Approaches to Perebor (Brute-Force Searches) Algorithms , 1984, Annals of the History of Computing.

[35]  V. Klee,et al.  Mathematical Programming and Convex Geometry , 1993 .

[36]  Yinyu Ye,et al.  A Short-Cut Potential Reduction Algorithm for Linear Programming , 1993 .

[37]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[38]  S. Janson.  Random coverings in several dimensions , 1986 .

[39]  Yinyu Ye.  Combining Binary Search and Newton's Method to Compute Real Roots for a Class of Real Functions , 1994, J. Complex..

[40]  Robert M. Freund,et al.  Some characterizations and properties of the “distance to ill-posedness” and the condition measure of a conic linear system , 1999, Math. Program..

[41]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[42]  J. Neumann,et al.  Numerical inverting of matrices of high order , 1947 .

[43]  Kurt M. Anstreicher.  On Long Step Path Following and SUMT for Linear and Quadratic Programming , 1996, SIAM J. Optim..

[44]  H. Heinrich,et al.  Goldstine, H. H., A History of Numerical Analysis from the 16th through the 19th Century. (Studies in the History of Mathematics and Physical Sciences 2) Berlin‐Heidelberg‐New York, Springer‐Verlag 1977. XIV, 348 S., 23 Abb., DM 54,–. US $27.00 , 1980 .

[45]  Jorge R. Vera,et al.  On the complexity of linear programming under finite precision arithmetic , 1998, Math. Program..

[46]  D. O’Leary.  On bounds for scaled projections and pseudoinverses , 1990 .

[47]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[48]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[49]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[50]  Karl-Heinz Borgwardt,et al.  The Average number of pivot steps required by the Simplex-Method is polynomial , 1982, Z. Oper. Research.

[51]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[52]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[53]  Gene H. Golub,et al.  Matrix computations , 1983 .

[54]  Michael A. Saunders,et al.  On projected newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method , 1986, Math. Program..

[55]  Renato D. C. Monteiro,et al.  Interior path following primal-dual algorithms. part I: Linear programming , 1989, Math. Program..

[56]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[57]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[58]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[59]  S. Vavasis.  Nonlinear optimization: complexity issues , 1991 .

[60]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[61]  Michael J. Todd,et al.  Characterizations, bounds, and probabilistic analysis of two complexity measures for linear programming problems , 2001, Math. Program..

[62]  H. L. Le Roy,et al.  Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; Vol. IV , 1969 .

[63]  Shinji Mizuno,et al.  A polynomial-time algorithm for a class of linear complementarity problems , 1989, Math. Program..

[64]  Y. Ye,et al.  Condition numbers for polyhedra with real number data , 1995, Oper. Res. Lett..

[65]  Anders Forsgren.  On Linear Least-Squares Problems with Diagonally Dominant Weight Matrices , 1996, SIAM J. Matrix Anal. Appl..

[66]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[67]  Yinyu Ye,et al.  On affine scaling algorithms for nonconvex quadratic programming , 1992, Math. Program..

[68]  N. Z. Shor.  Cut-off method with space extension in convex programming problems , 1977, Cybernetics.

[69]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[70]  Michael J. Todd,et al.  Infeasible-start primal-dual methods and infeasibility detectors for nonlinear programming problems , 1999, Math. Program..

[71]  Krzysztof C. Kiwiel,et al.  Exact penalty functions in proximal bundle methods for constrained convex nondifferentiable minimization , 1991, Math. Program..

[72]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[73]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[74]  Javier Peña,et al.  Understanding the Geometry of Infeasible Perturbations of a Conic Linear System , 1999, SIAM J. Optim..

[75]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[76]  M. H. Wright.  Numerical methods for nonlinearly constrained optimization , 1976 .

[77]  Leonid Khachiyan,et al.  On the Complexity of Approximating Extremal Determinants in Matrices , 1995, J. Complex..

[78]  Kaoru Tone,et al.  An active-set strategy in an interior point method for linear programming , 1991, Math. Program..

[79]  Joseph F. Traub,et al.  Computational Complexity of Iterative Processes , 1972, SIAM J. Comput..

[80]  J. Edmonds.  Systems of distinct representatives and linear algebra , 1967 .