The interior-point revolution in optimization: History, recent developments, and lasting consequences

Interior methods are a pervasive feature of the optimization landscape today, but it was not always so. Although interior-point techniques, primarily in the form of barrier methods, were widely used during the 1960s for problems with nonlinear constraints, their use for the fundamental problem of linear programming was unthinkable because of the total dominance of the simplex method. During the 1970s, barrier methods were superseded, nearly to the point of oblivion, by newly emerging and seemingly more efficient alternatives such as augmented Lagrangian and sequential quadratic programming methods. By the early 1980s, barrier methods were almost universally regarded as a closed chapter in the history of optimization. This picture changed dramatically in 1984, when Narendra Karmarkar announced a fast polynomial-time interior method for linear programming; in 1985, a formal connection was established between his method and classical barrier methods. Since then, interior methods have continued to transform both the theory and practice of constrained optimization. We present a condensed, unavoidably incomplete look at classical material and recent research about interior methods.

[1]  John Milnor,et al.  GROUPS WHICH ACT ON Su WITHOUT FIXED POINTS. , 1957 .

[2]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[3]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[4]  W. Murray,et al.  Analytical expressions for the eigenvalues and eigenvectors of the Hessian matrices of barrier and penalty functions , 1971 .

[5]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[6]  Philip E. Gill,et al.  Practical optimization , 1981 .

[7]  R. Hamilton.  Three-manifolds with positive Ricci curvature , 1982 .

[8]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[9]  Michael A. Saunders,et al.  On projected newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method , 1986, Math. Program..

[10]  David K. Smith.  Theory of Linear and Integer Programming , 1987 .

[11]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[12]  Johan Pas,et al.  Uniform p-adic cell decomposition and local zeta functions. , 1989 .

[13]  M. Todd,et al.  Chapter II Linear programming , 1989 .

[14]  Philip E. Gill,et al.  Numerical Linear Algebra and Optimization , 1991 .

[15]  Stig-Inge Gustafsson,et al.  Linear programming optimization in CHP networks , 1991 .

[16]  V. Kvetan,et al.  Parenteral nutrition and oral intake: effect of glucose and fat infusions. , 1991, JPEN. Journal of parenteral and enteral nutrition.

[17]  Margaret H. Wright,et al.  Interior methods for constrained optimization , 1992, Acta Numerica.

[18]  Clóvis C. Gonzaga,et al.  Path-Following Methods for Linear Programming , 1992, SIAM Rev..

[19]  Gautam Appa,et al.  Numerical Linear Algebra and Optimization: Volume 1 , 1992 .

[20]  R. Hamilton.  The Harnack estimate for the Ricci flow , 1993 .

[21]  P. Toint,et al.  A note on using alternative second-order models for the subproblems arising in barrier function methods for minimization , 1994 .

[22]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[23]  R. D. Murphy,et al.  Iterative solution of nonlinear equations , 1994 .

[24]  Margaret H. Wright,et al.  Some properties of the Hessian of the logarithmic barrier function , 1994, Math. Program..

[25]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[26]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[27]  Margaret H. Wright,et al.  Why a Pure Primal Newton Barrier Step May be Infeasible , 1995, SIAM J. Optim..

[28]  G. Perelman.  Spaces with Curvature Bounded Below , 1995 .

[29]  Stephen J. Wright.  Stability of Linear Equations Solvers in Interior-Point Methods , 1995, SIAM J. Matrix Anal. Appl..

[30]  Alexander Shapiro,et al.  On Eigenvalue Optimization , 1995, SIAM J. Optim..

[31]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[32]  T. Tsuchiya,et al.  On the formulation and theory of the Newton interior-point method for nonlinear programming , 1996 .

[33]  Joseph R. Shinnerl,et al.  Stability of Symmetric Ill-Conditioned Systems Arising in Interior Methods for Constrained Optimization , 1996, SIAM J. Matrix Anal. Appl..

[34]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[35]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[36]  R. Hamilton.  Four-manifolds with positive isotropic curvature , 1997 .

[37]  Motivic Igusa zeta functions , 1998, math/9803040.

[38]  Michael L. Overton,et al.  Primal-Dual Interior-Point Methods for Semidefinite Programming: Convergence Rates, Stability and Numerical Results , 1998, SIAM J. Optim..

[39]  Jean-Philippe Vial,et al.  Theory and algorithms for linear optimization - an interior point approach , 1998, Wiley-Interscience series in discrete mathematics and optimization.

[40]  Michael L. Overton,et al.  A Primal-dual Interior Method for Nonconvex Nonlinear Programming , 1998 .

[41]  Robert J. Vanderbei,et al.  Linear Programming: Foundations and Extensions , 1998, Kluwer international series in operations research and management service.

[42]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[43]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[44]  Anders Forsgren,et al.  Primal-Dual Interior Methods for Nonconvex Nonlinear Programming , 1998, SIAM J. Optim..

[45]  William Fulton,et al.  Intersection theory, Second Edition , 1998, Ergebnisse der Mathematik und ihrer Grenzgebiete.

[46]  Margaret H. Wright,et al.  Ill-Conditioning and Computational Error in Interior Methods for Nonlinear Programming , 1998, SIAM J. Optim..

[47]  Stephen J. Wright.  Modified Cholesky Factorizations in Interior-Point Algorithms for Linear Programming , 1999, SIAM J. Optim..

[48]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[49]  R. Hamilton.  Non-singular solutions of the Ricci flow on three-manifolds , 1999 .

[50]  J. Denef,et al.  Germs of arcs on singular algebraic varieties and motivic integration , 1999 .

[51]  J. Denef,et al.  Definable sets, motives and p-adic integrals , 1999, math/9910107.

[52]  E. Looijenga.  Motivic measures , 2000, math/0006220.

[53]  Jorge Nocedal,et al.  A trust region method based on interior point techniques for nonlinear programming , 2000, Math. Program..

[54]  Takao Yamaguchi,et al.  Collapsing Three-Manifolds Under a Lower Curvature Bound , 2000 .

[55]  P. Toint,et al.  A primal-dual algorithm for minimizing a non-convex function subject to bound and linear equality constraints , 2000 .

[56]  Nicholas I. M. Gould,et al.  A primal-dual trust-region algorithm for non-convex nonlinear programming , 2000, Math. Program..

[57]  F. Loeser,et al.  Motivic integration on smooth rigid varieties and invariants of degenerations , 2001, math/0107134.

[58]  Shang-Hua Teng,et al.  Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time , 2001, STOC '01.

[59]  Stephen J. Wright.  Effects of Finite-Precision Arithmetic on Interior-Point Methods for Nonlinear Programming , 2001, SIAM J. Optim..

[60]  Integration motivique sur les schemas formels , 2001, math/0112249.

[61]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[62]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[63]  G. Perelman.  The entropy formula for the Ricci flow and its geometric applications , 2002, math/0211159.

[64]  On some rational generating series occuring in arithmetic geometry , 2002, math/0212202.

[65]  Robert E. Bixby,et al.  Solving Real-World Linear Programs: A Decade and More of Progress , 2002, Oper. Res..

[67]  G. Perelman.  Finite extinction time for the solutions to the Ricci flow on certain three-manifolds , 2003, math/0307245.

[68]  Volume collapsed three-manifolds with a lower curvature bound , 2003, math/0304472.

[69]  G. Perelman.  Ricci flow with surgery on three-manifolds , 2003, math/0303109.

[70]  E. Lupercio,et al.  The Global Mckay–Ruan Correspondence Via Motivic Integration , 2003, math/0308200.

[71]  An introduction to motivic integration , 1999, math/9911179.

[72]  F. Loeser,et al.  Fonctions constructibles et intégration motivique I , 2004, math/0403349.