Dual techniques for constrained optimization

Algorithms to solve constrained optimization problems are derived. These schemes combine an unconstrained minimization scheme like the conjugate gradient method, an augmented Lagrangian, and multiplier updates to obtain global quadratic convergence. Since an augmented Lagrangian can be ill conditioned, a preconditioning strategy is developed to eliminate the instabilities associated with the penalty term. A criterion for deciding when to increase the penalty is presented.

[1]  Karl Löwner.  Über monotone Matrixfunktionen , 1934 .

[2]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[3]  Adi Ben-Israel.  A Newton-Raphson method for the solution of systems of equations , 1966 .

[4]  M. Powell.  A method for nonlinear constraints in minimization problems , 1969 .

[5]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[6]  A. Miele,et al.  Sequential gradient-restoration algorithm for optimal control problems , 1970 .

[7]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[8]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[9]  S. M. Robinson,et al.  A quadratically-convergent algorithm for general nonlinear programming problems , 1972, Math. Program..

[10]  J. B. Rosen,et al.  A Quadratically Convergent Lagrangian Algorithm for Nonlinear Constraints , 1972 .

[11]  R. Rockafellar.  The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[12]  A. Miele,et al.  Sequential gradient-restoration algorithm for optimal control problems with nondifferential constraints , 1974 .

[13]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[14]  V. T. Polyak.  TRET''''YAKOV: The method of penalty estimates for conditional extremum problems , 1974 .

[15]  F. Brezzi.  On the existence, uniqueness and approximation of saddle-point problems arising from lagrangian multipliers , 1974 .

[16]  R. Rockafellar.  Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming , 1974 .

[17]  Shih-Ping Han,et al.  Superlinearly convergent variable metric algorithms for general nonlinear programming problems , 1976, Math. Program..

[18]  R. Tapia.  Diagonalized multiplier methods and quasi-Newton methods for constrained optimization , 1977 .

[19]  A. Miele,et al.  Sequential gradient-restoration algorithm for optimal control problems with general boundary conditions , 1978 .

[20]  J. B. Rosen.  TWO-PHASE ALGORITHM FOR NONLINEAR CONSTRAINT PROBLEMS11This research was supported in part by the National Science Foundation grant MCS 76-23311. Support by the Systems Optimization Lab, Department of Operations Research, Stanford University, during the author's sabbatical leave, is also gratefully , 1978 .

[21]  M. J. D. Powell,et al.  Algorithms for nonlinear constraints that use lagrangian functions , 1978, Math. Program..

[22]  W. Hager.  Lipschitz Continuity for Constrained Processes , 1979 .

[23]  A. R. Mitchell,et al.  The Finite Difference Method in Partial Differential Equations , 1980 .

[24]  V. Klema.  LINPACK user's guide , 1980 .

[25]  Hiroshi Yamashita,et al.  A globally convergent constrained quasi-Newton method with an augmented lagrangian type penalty function , 1982, Math. Program..

[26]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[27]  Gene H. Golub,et al.  Matrix computations , 1983 .

[28]  W. Hager,et al.  Dual Approximations in Optimal Control , 1984 .

[29]  W. Hager.  Dual techinques for constrained optimization , 1985, 1985 24th IEEE Conference on Decision and Control.

[30]  W. Hager.  Approximations to the Multiplier Method , 1985 .

[31]  W. Hager.  Dual techniques for constrained optimization. II , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.