Nonmonotone trust region method for solving optimization problems

In this paper a trust region (TR) method with nonmonotone technique for optimization is proposed. We construct a new ratio of actual descent and predicted descent which is a simple and natural generalization of the modified Armijo line search rule. The paper exposes the relationship between the trust region method and line search approach. Since this method possesses the robust properties of trust region subproblem, it is globally convergent although we employ the nonmonotone sequence of function values instead of the monotone sequence. In addition, the proof of convergence is obviously simpler than one of Newton-type method with nonmonotone line search. Finally, applications of the nonmonotone TR algorithm to some optimization problems are discussed.

[1]  Ya-Xiang Yuan,et al.  A trust region algorithm for equality constrained optimization , 1990, Math. Program..

[2]  Martin Grötschel,et al.  Mathematical Programming The State of the Art, XIth International Symposium on Mathematical Programming, Bonn, Germany, August 23-27, 1982 , 1983, ISMP.

[3]  Defeng Sun,et al.  Global convergece of the bfgs algorithm with nonmonotone linesearch ∗ ∗this work is supported by national natural science foundation$ef: , 1995 .

[4]  Ya-Xiang Yuan,et al.  Conditions for convergence of trust region algorithms for nonsmooth optimization , 1985, Math. Program..

[5]  Richard H. Byrd,et al.  A Trust Region Algorithm for Nonlinearly Constrained Optimization , 1987 .

[6]  R. Fletcher.  A model algorithm for composite nondifferentiable optimization problems , 1982 .

[7]  Chengxian Xu,et al.  Trust region dogleg path algorithms for unconstrained minimization , 1999, Ann. Oper. Res..

[8]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[9]  Wenyu Sun,et al.  Trust region algorithm for nonsmooth optimization , 1997 .

[10]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[11]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[12]  J. Pang,et al.  A Trust Region Method for Constrained Nonsmooth Equations , 1994 .

[13]  S. Goldfeld,et al.  Maximization by Quadratic Hill-Climbing , 1966 .

[14]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[15]  Ya-Xiang Yuan,et al.  On a subproblem of trust region algorithms for constrained optimization , 1990, Math. Program..

[16]  Philippe L. Toint,et al.  An Assessment of Nonmonotone Linesearch Techniques for Unconstrained Optimization , 1996, SIAM J. Sci. Comput..

[17]  Jin Yun Yuan,et al.  Quasi-Newton trust region algorithm for non-smooth least squares problems , 1999, Appl. Math. Comput..

[18]  J. Z. Zhang,et al.  Projected quasi-Newton algorithm with trust region for constrained optimization , 1990 .

[19]  Roger Fletcher,et al.  An algorithm for solving linearly constrained optimization problems , 1972, Math. Program..

[20]  A. Vardi.  A Trust Region Algorithm for Equality Constrained Minimization: Convergence Properties and Implementation , 1985 .

[21]  C. Lemaréchal,et al.  The watchdog technique for forcing convergence in algorithms for constrained optimization , 1982 .

[22]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[23]  W. Hager,et al.  Large Scale Optimization : State of the Art , 1993 .

[24]  Wenyu Sun,et al.  Global convergence of nonmonotone descent methods for unconstrained optimization problems , 2002 .

[25]  Richard A. Tapia,et al.  A unified approach to global convergence of trust region methods for nonsmooth optimization , 1995, Math. Program..

[26]  N. Deng,et al.  Nonmonotonic trust region algorithm , 1993 .

[27]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[28]  M. J. D. Powell,et al.  On the global convergence of trust region algorithms for unconstrained minimization , 1984, Math. Program..

[29]  Wenyu Sun,et al.  A trust region method for conic model to solve unconstraind optimizaions , 1996 .

[30]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[31]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[32]  Jong-Shi Pang,et al.  Minimization of Locally Lipschitzian Functions , 1991, SIAM J. Optim..

[33]  Liqun Qi,et al.  A trust region algorithm for minimization of locally Lipschitzian functions , 1994, Math. Program..

[34]  L. Grippo,et al.  A truncated Newton method with nonmonotone line search for unconstrained optimization , 1989 .