On the use of quadratic models in unconstrained minimization without derivatives

Quadratic approximations to the objective function provide a way of estimating first and second derivatives in iterative algorithms for unconstrained minimization. Therefore, we address the construction of suitable quadratic models Q by interpolating values of the objective function F. On a typical iteration, the objective function is calculated at the point that minimizes the current quadratic model subject to a trust region bound, and we find that these values of F provide good information for the updating of Q, except that a few extra values are needed occasionally to avoid degeneracy. The number of interpolation points and their positions can be controlled adequately by deleting one of the current points to make room for each new one. An algorithm is described that works in this way. It is applied to an optimization calculation that has between 10 and 160 variables. The numerical results suggest that, if m = 2n + 1, then the number of evaluations of F is only of magnitude n, where m and n are the number of interpolation conditions of each model and the number of variables, respectively. This success is due to the technique that updates Q. It minimizes the Frobenius norm of the change to ∇2 Q, subject to the interpolation conditions that have been mentioned.