A dimension-reducing method for unconstrained optimization

Abstract A new method for unconstrained optimization in R n is presented. This method reduces the dimension of the problem in such a way that it can lead to an iterative approximate formula for the computation of (n − 1) components of the optimum while its remaining component is computed separately using the final approximations of the other components. It converges quadratically to a local optimum and it requires storage of order (n − 1) × (n − 1). Besides, it does not require a good initial guess for one component of the optimum and it does not directly perform gradient evaluations; thus it can be applied to problems with imprecise gradient values. Moreover, a procedure for transforming the matrix formed by our method into a symmetric as well as into a diagonal one is presented. Furthermore, the proposed dimension-reducing scheme using finite difference gradient and Hessian is presented. The methods have been implemented and tested. Performance information for well-known test functions is reported.

[1]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[2]  C. Botsaris,et al.  A new conic method for unconstrained minimization , 1992 .

[3]  Michael N. Vrahatis,et al.  Algorithm 666: Chabis: a mathematical software package for locating and evaluating roots of systems of nonlinear equations , 1988, TOMS.

[4]  K. Sikorski.  Bisection is optimal , 1982 .

[5]  J. Traub.  Iterative Methods for the Solution of Equations , 1982 .

[6]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[7]  Kenneth M. Brown,et al.  COMPUTER ORIENTED ALGORITHMS FOR SOLVING SYSTEMS OF SIMULTANEOUS NONLINEAR ALGEBRAIC EQUATIONS**This work was supported in part by the National Science Foundation under Grant GJ-32552 and in part by the University Computer Center of the University of Minnesota. , 1973 .

[8]  George D. Byrne,et al.  Numerical solution of systems of nonlinear algebraic equations : papers , 1973 .

[9]  Michael N. Vrahatis,et al.  A new dimension-reducing method for solving systems of nonlinear equations , 1990 .

[10]  William H. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[11]  M. N. Vrahatis,et al.  A New Unconstrained Optimization Method for Imprecise Function and Gradient Values , 1996 .

[12]  M N Vrahatis,et al.  A rapid Generalized Method of Bisection for solving Systems of Non-linear Equations , 1986 .

[13]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[14]  William H. Press,et al.  Numerical recipes , 1990 .

[15]  R. Brent.  Some Efficient Algorithms for Solving Systems of Nonlinear Equations , 1973 .

[16]  Michael N. Vrahatis,et al.  Solving systems of nonlinear equations using the nonzero value of the topological degree , 1988, TOMS.

[17]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[18]  Michael N. Vrahatis,et al.  The implicit function theorem for solving systems of nonlinear equations in , 1989 .

[19]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[20]  R. Baker Kearfott,et al.  Some tests of generalized bisection , 1987, TOMS.

[21]  J. Ecker,et al.  A note on solution of nonlinear programming problems with imprecise function and gradient values , 1987 .

[22]  J. Miller.  Numerical Analysis , 1966, Nature.

[23]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[24]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[25]  A. Ostrowski.  Solution of equations in Euclidean and Banach spaces , 1973 .

[26]  Michael N. Vrahatis,et al.  A short proof and a generalization of Miranda’s existence theorem , 1989 .

[27]  J. Dieudonne.  Foundations of Modern Analysis , 1969 .

[28]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[29]  R. D. Murphy,et al.  Iterative solution of nonlinear equations , 1994 .

[30]  T.,et al.  SOLVING SYSTEMS OF NONLINEAR EQUATIONS IN Rn USING A ROTATING HYPERPLANE IN Rn " , 1989 .

[31]  Michael N. Vrahatis,et al.  Locating and Computing All the Simple Roots and Extrema of a Function , 1996, SIAM J. Sci. Comput..

[32]  D. Jacobson,et al.  An algorithm that minimizes homogeneous functions of n variables in n+2 iterations and rapidly minim , 1972 .

[33]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[34]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[35]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[36]  Michael N. Vrahatis,et al.  Solving systems of nonlinear equations In using a rotating hyperplane in , 1990, Int. J. Comput. Math..

[37]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..