In this paper, the problem of minimizing a functionf(x) subject to a constraint ϕ(x)=0 is considered. Here,f is a scalar,x ann-vector, and ϕ aq-vector, withq<n. The use of the augmented penalty function is explored in connection with theconjugate gradient-restoration algorithm. The augmented penalty functionW(x, λ,k) is defined to be the linear combination of the augmented functionF(x, λ) and the constraint errorP(x), where theq-vector λ is the Lagrange multiplier and the scalark is the penalty constant.The conjugate gradient-restoration algorithm includes a conjugate-gradient phase involvingn-q iterations and a restoration phase involving one iteration. In the conjugate-gradient phase, one tries to improve the value of the function, while avoiding excessive constraint violation. In the restoration phase, one reduces the constraint error, while avoiding excessive change in the value of the function.Concerning the conjugate-gradient phase, two classes of algorithms are considered: for algorithms of Class I, the Lagrange multiplier λ is determined so that the error in the optimum condition is minimized for givenx; for algorithms of Class II, the Lagrange multiplier λ is determined so that the constraint is satisfied to first order. For each class, two versions are studied. In version (α), the penalty constant is held unchanged throughout the entire algorithm. In version (β), the penalty constant is updated at the beginning of each conjugate-gradient phase so as to achieve certain desirable properties.Concerning the restoration phase, the minimum distance algorithm is employed. Since the use of the augmented penalty function automatically prevents excessive constraint violation, single-step restoration is considered.If the functionf(x) is quadratic and the constraint ϕ(x) is linear, all the previous algorithms are identical, that is, they produce the same sequence of points and converge to the solution in the same number of iterations. This number of iterations is at mostN*=n−q if the starting pointxs is such that ϕ(xs)=0 and at mostN*=1+n−q if the starting pointxs is such that ϕ(xs)≠0.In order to illustrate the theory, seven numerical examples are developed. The first example refers to a quadratic function and a linear constraint. The remaining examples refer to nonquadratic functions and nonlinear constraints. For the linear-quadratic example, all the algorithms behave identically, as predicted by the theory. For the nonlinear-nonquadratic examples, algorithms of Class II generally exhibit faster convergence than algorithms of Class I, and algorithms of type (β) generally exhibit faster convergence than algorithms of type (α).
[1]
M. Hestenes.
Multiplier and gradient methods
,
1969
.
[2]
J. C. Heideman,et al.
Sequential gradient-restoration algorithm for the minimization of constrained functions—Ordinary and conjugate gradient versions
,
1969
.
[3]
P. C. Haarhoff,et al.
A New Method for the Optimization of a Nonlinear Function Subject to Nonlinear Constraints
,
1970,
Comput. J..
[4]
A. V. Levy,et al.
Use of the augmented penalty function in mathematical programming problems, part 1
,
1971
.
[5]
A. V. Levy,et al.
Modifications and extensions of the conjugate gradient-restoration algorithm for mathematical programming problems
,
1971
.