Solving structured nonlinear least-squares and nonlinear feasibility problems with expensive functions

We present an algorithm for nonlinear least-squares and nonlinear feasibility problems, i.e. for systems of nonlinear equations and nonlinear inequalities, which depend on the outcome of expensive functions for which derivatives are assumed to be unavailable. Our algorithm combines derivativefree techniques with filter trust-region methods to keep the number of expensive function evaluations low and to obtain a robust method. Under adequate assumptions, we show global convergence to a feasible point. Numerical results indicate a significant reduction in function evaluations compared to other derivative based and derivative-free solvers for nonlinear feasibility problems.

[1]  Nicholas I. M. Gould,et al.  A Filter-Trust-Region Method for Unconstrained Optimization , 2005, SIAM J. Optim..

[2]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[3]  Benedetta Morini,et al.  TRESNEI, a Matlab trust-region solver for systems of nonlinear equalities and inequalities , 2012, Comput. Optim. Appl..

[4]  Chih-Jen Lin,et al.  Newton's Method for Large Bound-Constrained Optimization Problems , 1999, SIAM J. Optim..

[5]  P. Toint,et al.  Testing a class of methods for solving minimization problems with simple bounds on the variables , 1988 .

[6]  Jorge J. Moré,et al.  User Guide for Minpack-1 , 1980 .

[7]  Benoît Colson,et al.  Trust-region algorithms for derivative-free optimization and nonlinear bilevel programming , 2004, 4OR.

[8]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[9]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[10]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[11]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[12]  Caroline Sainvitu,et al.  A filter-trust-region method for simple-bound constrained optimization , 2007, Optim. Methods Softw..

[13]  Nicholas I. M. Gould,et al.  Global Convergence of a Trust-Region SQP-Filter Algorithm for General Nonlinear Programming , 2002, SIAM J. Optim..

[14]  Pu-yan Nie.  A derivative-free method for the system of nonlinear equations , 2006 .

[15]  Nicholas I. M. Gould,et al.  A Multidimensional Filter Algorithm for Nonlinear Equations and Nonlinear Least-Squares , 2004, SIAM J. Optim..

[16]  Serge Gratton,et al.  Recursive Trust-Region Methods for Multiscale Nonlinear Optimization , 2008, SIAM J. Optim..

[17]  Philip E. Gill,et al.  Iterative Methods for Finding a Trust-region Step , 2009, SIAM J. Optim..

[18]  Benedetta Morini,et al.  Trust-region quadratic methods for nonlinear systems of mixed equalities and inequalities , 2009 .

[19]  J. M. Martínez,et al.  Algorithms for Solving Nonlinear Systems of Equations , 1994 .

[20]  Nicholas I. M. Gould,et al.  FILTRANE, a Fortran 95 filter-trust-region package for solving nonlinear least-squares and nonlinear feasibility problems , 2007, TOMS.

[21]  Gerardo Toraldo,et al.  On the Solution of Large Quadratic Programming Problems with Bound Constraints , 1991, SIAM J. Optim..

[22]  Jorge J. Moré,et al.  Trust regions and projected gradients , 1988 .

[23]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[24]  J. E. DENNIS,et al.  A Trust-Region Approach to Nonlinear Systems of Equalities and Inequalities , 1999, SIAM J. Optim..

[25]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[26]  Ernst Rank,et al.  A Multigrid-Solver for the Discrete Boltzmann Equation , 2002 .

[27]  Ajith Abraham,et al.  A New Approach for Solving Nonlinear Equations Systems , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[28]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[29]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[30]  E. Spedicato.  Algorithms for continuous optimization : the state of the art , 1994 .

[31]  Katya Scheinberg,et al.  Geometry of interpolation sets in derivative free optimization , 2007, Math. Program..