Minimizing a differentiable function over a differential manifold

To generalize the descent methods of unconstrained optimization to the constrained case, we define intrinsically the gradient field of the objective function on the constraint manifold and analyze descent methods along geodesics, including the gradient projection and reduced gradient methods for special choices of coordinate systems. In particular, we generalize the quasi-Newton methods and establish their superlinear convergence; we show that they only require the updating of a reduced size matrix. In practice, the geodesic search is approximated by a tangent step followed by a constraints restoration or by a simple arc search again followed by a restoration step.

[1]  J. B. Rosen.  The Gradient Projection Method for Nonlinear Programming. Part I. Linear Constraints , 1960 .

[2]  J. B. Rosen.  The gradient projection method for nonlinear programming: Part II , 1961 .

[3]  R. Bishop,et al.  Geometry of Manifolds , 1964 .

[4]  N. Hicks.  Notes on Differential Geometry , 1967 .

[5]  D. Goldfarb.  Extension of Davidon’s Variable Metric Method to Maximization Under Linear Inequality and Equality Constraints , 1969 .

[6]  J. C. Heideman,et al.  Sequential gradient-restoration algorithm for the minimization of constrained functions—Ordinary and conjugate gradient versions , 1969 .

[7]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[8]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[9]  D. Luenberger.  The Gradient Projection Method Along Geodesics , 1972 .

[10]  P. Gill,et al.  Quasi-Newton Methods for Unconstrained Optimization , 1972 .

[11]  G. Stewart.  Introduction to matrix computations , 1973 .

[12]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[13]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[14]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[15]  M. Hestenes.  Optimization Theory: The Finite Dimensional Case , 1975 .

[16]  D. Luenberger,et al.  Efficiently Converging Minimization Methods Based on the Reduced Gradient , 1976 .

[17]  Garth P. McCormick,et al.  A modification of Armijo's step-size rule for negative curvature , 1977, Math. Program..

[18]  E. Polak,et al.  On the Use of Approximations in Algorithms for Optimization Problems with Equality and Inequality Constraints , 1978 .

[19]  Danny C. Sorensen,et al.  On the use of directions of negative curvature in a modified newton method , 1979, Math. Program..

[20]  K. Tanabe.  A geometric method in nonlinear programming , 1980 .

[21]  D. Gabay.  Reduced quasi-Newton methods with feasibility improvement for nonlinearly constrained optimization , 1982 .