Modification of DIRECT for high-dimensional design problems

DIviding RECTangles (DIRECT), as a well-known derivative-free global optimization method, has been found to be effective and efficient for low-dimensional problems. When facing high-dimensional black-box problems, however, DIRECT's performance deteriorates. This work proposes a series of modifications to DIRECT for high-dimensional problems (dimensionality d>10). The principal idea is to increase the convergence speed by breaking its single initialization-to-convergence approach into several more intricate steps. Specifically, starting with the entire feasible area, the search domain will shrink gradually and adaptively to the region enclosing the potential optimum. Several stopping criteria have been introduced to avoid premature convergence. A diversification subroutine has also been developed to prevent the algorithm from being trapped in local minima. The proposed approach is benchmarked using nine standard high-dimensional test functions and one black-box engineering problem. All these tests show a significant efficiency improvement over the original DIRECT for high-dimensional design problems.

[1]  Herschel Rabitz,et al.  Efficient Implementation of High Dimensional Model Representations , 2001 .

[2]  Blaine Lilly,et al.  Mechanical Assemblies: their Design, Manufacture, and Role in Product Development , 2013 .

[3]  D. Finkel,et al.  Convergence analysis of the direct algorithm , 2004 .

[4]  C. T. Kelley,et al.  A Locally-Biased form of the DIRECT Algorithm , 2001, J. Glob. Optim..

[5]  Xin‐She Yang,et al.  Appendix A: Test Problems in Optimization , 2010 .

[6]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[7]  E. L. Lawler,et al.  Branch-and-Bound Methods: A Survey , 1966, Oper. Res..

[8]  Michael C. Ferris,et al.  Extension of the direct optimization algorithm for noisy functions , 2007, 2007 Winter Simulation Conference.

[9]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[10]  G. G. Wang,et al.  Mode-pursuing sampling method for global optimization on expensive black-box functions , 2004 .

[11]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[12]  Xin-She Yang.  Test Problems in Optimization , 2010, 1008.0549.

[13]  Lakhdar Chiter.  A new sampling method in the DIRECT algorithm , 2006, Appl. Math. Comput..

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  C. T. Kelley,et al.  Modifications of the direct algorithm , 2001 .

[16]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[17]  Donald R. Jones,et al.  Direct Global Optimization Algorithm , 2009, Encyclopedia of Optimization.

[18]  Lakhdar Chiter.  DIRECT algorithm: A new definition of potentially optimal hyperrectangles , 2006, Appl. Math. Comput..

[19]  Arnold Neumaier,et al.  Global Optimization by Multilevel Coordinate Search , 1999, J. Glob. Optim..

[20]  G. G. Wang,et al.  Metamodeling for High Dimensional Simulation-Based Design Problems , 2010 .

[21]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[22]  David B. Bogy,et al.  Hard disc drive air bearing design: modified DIRECT algorithm and its application to slider air bearing surface optimization , 2004 .

[23]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[24]  Alan F. Murray,et al.  IEEE International Conference on Neural Networks , 1997 .

[25]  Haitao Lang,et al.  Design of URAs by DIRECT global optimization algorithm , 2009 .

[26]  H. Rabitz,et al.  General foundations of high‐dimensional model representations , 1999 .

[27]  D. Eichmann.  More Test Examples For Nonlinear Programming Codes , 2016 .