Lagrange Multipliers and Optimality

Lagrange multipliers used to be viewed as auxiliary variables introduced in a problem of constrained minimization in order to write first-order optimality conditions formally as a system of equations. Modern applications, with their emphasis on numerical methods and more complicated side conditions than equations, have demanded deeper understanding of the concept and how it fits into a larger theoretical picture. A major line of research has been the nonsmooth geometry of one-sided tangent and normal vectors to the set of points satisfying the given constraints. Another has been the game-theoretic role of multiplier vectors as solutions to a dual problem. Interpretations as generalized derivatives of the optimal value with respect to problem parameters have also been explored. Lagrange multipliers are now being seen as arising from a general rule for the subdifferentiation of a nonsmooth objective function which allows black-and-white constraints to be replaced by penalty expressions. This paper traces su...

[1]  J. Farkas.  Theorie der einfachen Ungleichungen. , 1902 .

[2]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[3]  E. Neville,et al.  Introduction a la Geometrie Infinitesimale Directe , 1933 .

[4]  R. Courant.  Variational methods for the solution of problems of equilibrium and vibrations , 1943 .

[5]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[6]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .

[7]  L. Pennisi.  An Indirect Sufficiency Proof for the Problem of Lagrange with Differential Inequalities as Added Side Conditions , 1953 .

[8]  W. Zangwill.  Non-Linear Programming Via Penalty Functions , 1967 .

[9]  Douglas Gale.  A Geometric Duality Theorem with Economic Applications , 1967 .

[10]  E. Beltrami.  A constructive proof of the Kuhn-Tucker multiplier rule , 1969 .

[11]  M. Powell.  A method for nonlinear constraints in minimization problems , 1969 .

[12]  T. Pietrzykowski.  An Exact Potential Method for Constrained Maxima , 1969 .

[13]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[14]  R. Rockafellar.  Some Convex Programs Whose Duals Are Linearly Constrained , 1970 .

[15]  R. Rockafellar.  Ordinary convex programs without a duality gap , 1971 .

[16]  R. Rockafellar.  The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[17]  Floyd J. Gould,et al.  A general saddle point result for constrained optimization , 1973, Math. Program..

[18]  E. J. McShane.  The Lagrange Multiplier Rule , 1973 .

[19]  M. Nashed,et al.  On the cones of tangents with applications to mathematical programming , 1974 .

[20]  R. Rockafellar.  Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming , 1974 .

[21]  O. Mangasarian.  Unconstrained Lagrangians in Nonlinear Programming , 1975 .

[22]  F. Clarke.  Generalized gradients and applications , 1975 .

[23]  B. Mordukhovich.  Maximum principle in the problem of time optimal response with nonsmooth constraints PMM vol. 40, n≗ 6, 1976, pp. 1014-1023 , 1976 .

[24]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[25]  Frank H. Clarke,et al.  A New Approach to Lagrange Multipliers , 1976, Math. Oper. Res..

[26]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[27]  J. Penot.  Calcul sous-differentiel et optimisation , 1978 .

[28]  A. Ioffe.  Necessary and Sufficient Conditions for a Local Minimum. 3: Second Order Conditions and Augmented Duality , 1979 .

[29]  A. Ben-Tal.  Second-order and related extremality conditions in nonlinear programming , 1980 .

[30]  R. Tyrrell Rockafellar,et al.  Proximal Subgradients, Marginal Values, and Augmented Lagrangians in Nonconvex Optimization , 1981, Math. Oper. Res..

[31]  Harold W. Kuhn,et al.  Nonlinear programming: a historical view , 1982, SMAP.

[32]  R. Rockafellar.  Lagrange multipliers and subderivatives of optimal value functions in nonlinear programming , 1982 .

[33]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[34]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[35]  A. Ioffe.  Approximate subdifferentials and applications. I. The finite-dimensional theory , 1984 .

[36]  R. Rockafellar.  Directional differentiability of the optimal value function in a nonlinear programming problem , 1984 .

[37]  J. Aubin,et al.  Applied Nonlinear Analysis , 1984 .

[38]  A. Ioffe.  Calculus of Dini subdifferentials of functions and contingent coderivatives of set-valued maps , 1984 .

[39]  A. Barrett.  Network Flows and Monotropic Optimization. , 1984 .

[40]  R. Rockafellar.  Extensions of subgradient calculus with applications to optimization , 1985 .

[41]  R. Rockafellar,et al.  A Lagrangian Finite Generation Technique for Solving Linear-Quadratic Problems in Stochastic Programming , 1986 .

[42]  Roger J.-B. Wets,et al.  Linear-quadratic programming problems with stochastic penalties: The finite generation algorithm , 1986 .

[43]  R. Rockafellar.  Linear-quadratic programming and optimal control , 1987 .

[44]  R. Rockafellar.  Conjugate Duality and Optimization , 1987 .

[45]  R. Tyrrell Rockafellar,et al.  Second-Order Optimality Conditions in Nonlinear Programming Obtained by Way of Epi-Derivatives , 1989, Math. Oper. Res..

[46]  R. Rockafellar.  Perturbation of Generalized Kuhn-Tucker Points in Finite-Dimensional Optimization , 1989 .

[47]  R. Rockafellar.  Hamiltonian trajectories and duality in the optimal control of linear systems with convex costs , 1989 .

[48]  F. Clarke.  Methods of dynamic and nonsmooth optimization , 1989 .

[49]  R. Rockafellar,et al.  Generalized linear-quadratic problems of deterministic and stochastic optimal control in discrete time , 1990 .

[50]  R. Tyrrell Rockafellar,et al.  Computational schemes for large-scale problems in extended linear-quadratic programming , 1990, Math. Program..

[51]  R. Rockafellar.  Nonsmooth analysis and parametric optimization , 1990 .

[52]  J. Burke.  An exact penalization viewpoint of constrained optimization , 1991 .

[53]  R. Rockafellar,et al.  Amenable functions in optimization , 1992 .

[54]  J. V. Burke,et al.  Optimality conditions for non-finite valued convex composite functions , 1992, Math. Program..

[55]  R. Rockafellar,et al.  A Calculus of EPI-Derivatives Applicable to Optimization , 1993, Canadian Journal of Mathematics.

[56]  R. Tyrrell Rockafellar,et al.  Primal-Dual Projected Gradient Algorithms for Extended Linear-Quadratic Programming , 1993, SIAM J. Optim..

[57]  P. Berck,et al.  Calculus of variations and optimal control theory , 1993 .

[58]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .