A derivative-free trust-funnel method for equality-constrained nonlinear optimization

A new derivative-free method is proposed for solving equality-constrained nonlinear optimization problems. This method is of the trust-funnel variety and is also based on the use of polynomial interpolation models. In addition, it uses a self-correcting geometry procedure in order to ensure that the interpolation problem is well defined in the sense that the geometry of the set of interpolation points does not differ too much from the ideal one. The algorithm is described in detail and some encouraging numerical results are presented.Graphical Abstract

[1]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[2]  Katya Scheinberg,et al.  A derivative free optimization algorithm in practice , 1998 .

[3]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[4]  Robert Michael Lewis,et al.  Pattern Search Algorithms for Bound Constrained Minimization , 1999, SIAM J. Optim..

[5]  M. Powell.  The BOBYQA algorithm for bound constrained optimization without derivatives , 2009 .

[6]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[7]  Robert Michael Lewis,et al.  Pattern Search Methods for Linearly Constrained Minimization , 1999, SIAM J. Optim..

[8]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[9]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[10]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[11]  Serge Gratton,et al.  An active-set trust-region method for derivative-free nonlinear bound-constrained optimization , 2011, Optim. Methods Softw..

[12]  Katya Scheinberg,et al.  On the convergence of derivative-free methods for unconstrained optimization , 1997 .

[13]  G. W. Stewart,et al.  A Modification of Davidon's Minimization Method to Accept Difference Approximations of Derivatives , 1967, JACM.

[14]  Philip E. Gill,et al.  Practical optimization , 1981 .

[15]  Robert Michael Lewis,et al.  Active Set Identification for Linearly Constrained Minimization Without Explicit Derivatives , 2009, SIAM J. Optim..

[16]  E. Omojokun.  Trust region algorithms for optimization with nonlinear equality and inequality constraints , 1990 .

[17]  Katya Scheinberg,et al.  On the local convergence of a derivative-free algorithm for least-squares minimization , 2010, Computational Optimization and Applications.

[18]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[19]  Nicholas I. M. Gould,et al.  On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization , 2001, SIAM J. Sci. Comput..

[20]  M. Powell.  Developments of NEWUOA for minimization without derivatives , 2008 .

[21]  Nicholas I. M. Gould,et al.  Nonlinear programming without a penalty function or a filter , 2010, Math. Program..

[22]  Jorge Nocedal,et al.  On the geometry phase in model-based algorithms for derivative-free optimization , 2009, Optim. Methods Softw..

[23]  Katya Scheinberg,et al.  Self-Correcting Geometry in Model-Based Algorithms for Derivative-Free Unconstrained Optimization , 2010, SIAM J. Optim..

[24]  V. Torczon,et al.  A GLOBALLY CONVERGENT AUGMENTED LAGRANGIAN ALGORITHM FOR OPTIMIZATION WITH GENERAL CONSTRAINTS AND SIMPLE BOUNDS , 2002 .

[25]  V. Torczon,et al.  Direct search methods: then and now , 2000 .

[26]  M. J. D. Powell,et al.  Direct search algorithms for optimization calculations , 1998, Acta Numerica.

[27]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[28]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[29]  Katya Scheinberg,et al.  Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization , 2012, Mathematical Programming.

[30]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[31]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[32]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[33]  Benoît Colson,et al.  Trust-region algorithms for derivative-free optimization and nonlinear bilevel programming , 2004, 4OR.

[34]  M. J. D. Powell,et al.  Least Frobenius norm updating of quadratic models that satisfy interpolation conditions , 2004, Math. Program..

[35]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[36]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[37]  M. Powell.  A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[38]  D. Winfield,et al.  Function Minimization by Interpolation in a Data Table , 1973 .