Structured trust region algorithms for the minimization of nonlinear functions

Among nonlinear optimization approaches, the trust region approach is known to have excellent convergence properties. The idea is to lengthen or shorten the length of each iterative step depending on how well a quadratic model of the nonlinear function 'fits'. The step is computed as a minimizer of the quadratic model. Usual trust region algorithms maintain a single 'trust region radius' that restricts the step length. But there is no particular reason why the length of the step should be restricted equally in all directions. In particular, if the function to be minimized, and the constraints, show a type of structure called 'partial separability', then it is possible for the fit between the model and the function to differ in the various 'partially separable' subspaces. Thus it seems logical to restrict the size of the step in the more highly non-linear subspaces, and while allowing it to be longer along the more linear directions, so that longer steps may allow faster convergence in the end. Large problems often exhibit a partially separable structure. The main reference for this paper is one by Conn, Gould, Toint, 1991 (15). However, the nice convergence results that go with trust region methods do not carry over unless such a 'multiple trust region approach' is implemented appropriately. This dissertation discusses three approaches to controlling the sizes of the trust regions. We give first-order convergence results for the convex-constrained minimization problem for two of our algorithms, and for the unconstrained case for the third algorithm. We give second-order convergence results for the unconstrained case for all the algorithms. Finally we give implementation and computational results for the unconstrained problem.