On D-optimality based trust regions for black-box optimization problems

Various sequential derivative-free optimization algorithms exist for solving black-box optimization problems. Two important building blocks in these algorithms are the trust region and the geometry improvement. In this paper, we propose to incorporate the D-optimality criterion, well known in the design of experiments, into these algorithms in two different ways. Firstly, it is used to define a trust region that adapts its shape to the locations of the points in which the objective function has been evaluated. Secondly, it is used to determine an optimal geometry-improving point. The proposed trust region and geometry improvement can both be implemented into existing sequential algorithms.

[1]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[2]  M. J. Box,et al.  Factorial Designs, the |X′X| Criterion, and Some Related Matters , 1971 .

[3]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[4]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[5]  Dick den Hertog,et al.  Optimizing color picture tubes by high-cost nonlinear programming , 2002, Eur. J. Oper. Res..

[6]  O. Dykstra.  The Augmentation of Experimental Data to Maximize [X′X] , 1971 .

[7]  Toby J. Mitchell,et al.  An algorithm for the construction of “ D -optimal” experimental designs , 2000 .

[8]  Jorge Nocedal,et al.  Wedge trust region methods for derivative free optimization , 2002, Math. Program..

[9]  Y. Ye.  A new complexity result on minimization of a quadratic function with a sphere constraint , 1992 .

[10]  J. Kleijnen,et al.  Searching for the Maximum Output in Random Simulation : New Signal / Noise Heuristics , 2022 .

[11]  M. Powell.  A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[12]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[13]  Katya Scheinberg,et al.  Recent progress in unconstrained nonlinear optimization without derivatives , 1997, Math. Program..

[14]  Susana Gomez,et al.  Advances in Optimization and Numerical Analysis , 1994 .

[15]  Herbert Hamers,et al.  Constrained optimization involving expensive function evaluations: A sequential approach , 2005, Eur. J. Oper. Res..

[16]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[17]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[18]  P. Toint,et al.  An Algorithm using Quadratic Interpolation for Unconstrained Derivative Free Optimization , 1996 .