Automated Response Surface Model Generation with Sequential Design

The increasing use of expensive computer simulations in engineering places a serious computational burden on associated optimization problems. Surrogate modelling becomes standard practice in analyzing such expensive blackbox problems. Moreover, surrogate based optimization (SBO) is able to drastically reduce the number of needed function evaluations with respect to traditional methods. This paper briefly discusses several approaches available which use surrogate models for optimization and highlights one sequential design approach in particular, i.e., expected improvement. Expected improvement is described in detail, along with recent related work. The approach has been implemented in a readily available research platform for surrogate modelling and demonstrated on a concrete application from Electro-Magnetics (EM). The results hold competitive designs and one optimum is even able to outperform the reference optimum obtained using extensive domain specific knowledge.

[1]  Søren Nymand Lophaven,et al.  Aspects of the Matlab toolbox DACE , 2002 .

[2]  Robert J. Weber.  Microwave Filter Design , 2001 .

[3]  Marco Locatelli,et al.  Bayesian Algorithms for One-Dimensional Global Optimization , 1997, J. Glob. Optim..

[4]  William J. Welch,et al.  Computer experiments and global optimization , 1997 .

[5]  Yao Lin,et al.  An Efficient Robust Concept Exploration Method and Sequential Exploratory Experimental Design , 2004 .

[6]  G. Matheron.  Principles of geostatistics , 1963 .

[7]  Christine A. Shoemaker,et al.  Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions , 2005, J. Glob. Optim..

[8]  Michael James Sasena,et al.  Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. , 2002 .

[9]  Dirk Gorissen,et al.  Sequential modeling of a low noise amplifier with neural networks and active learning , 2009, Neural Computing and Applications.

[10]  Andy J. Keane,et al.  Multi-Objective Optimization Using Surrogates , 2010 .

[11]  Christopher M. Bishop,et al.  Bayesian Regression and Classification , 2003 .

[12]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[13]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[14]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[15]  M.C.E. Yagoub,et al.  Advanced microwave modeling framework exploiting automatic model generation, knowledge neural networks and space mapping , 2002, 2002 IEEE MTT-S International Microwave Symposium Digest (Cat. No.02CH37278).

[16]  Andy J. Keane,et al.  On the Design of Optimization Strategies Based on Global Response Surface Approximation Models , 2005, J. Glob. Optim..

[17]  Donald R. Jones,et al.  Global optimization of deceptive functions with sparse sampling , 2008 .

[18]  A. O'Hagan,et al.  Predicting the output from a complex computer code when fast approximations are available , 2000 .

[19]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[20]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[21]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[22]  Wolfgang Ponweiser,et al.  Clustered multiple generalized expected improvement: A novel infill sampling criterion for surrogate models , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[23]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[24]  V. R. Joseph,et al.  ORTHOGONAL-MAXIMIN LATIN HYPERCUBE DESIGNS , 2008 .

[25]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[26]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[27]  G. G. Wang,et al.  Mode-pursuing sampling method for global optimization on expensive black-box functions , 2004 .

[28]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[29]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[30]  Jack P. C. Kleijnen,et al.  The correct Kriging variance estimated by bootstrapping , 2006, J. Oper. Res. Soc..

[31]  Stephen J. Leary,et al.  A parallel updating scheme for approximating and optimizing high fidelity computer simulations , 2004 .

[32]  Robert B. Gramacy,et al.  Parameter space exploration with Gaussian process trees , 2004, ICML.

[33]  Melissa Ness.  Microwave Filter Design , 2000 .

[34]  Agus Sudjianto,et al.  Blind Kriging: A New Method for Developing Metamodels , 2008 .

[35]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[36]  M. Gibbs,et al.  Efficient implementation of gaussian processes , 1997 .

[37]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[38]  Andy J. Keane,et al.  Statistical Improvement Criteria for Use in Multiobjective Design Optimization , 2006 .

[39]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[40]  G. Gary Wang,et al.  An Efficient Pareto Set Identification Approach for Multiobjective Optimization on Black-Box Functions , 2005 .

[41]  Hirotaka Nakayama,et al.  Meta-Modeling in Multiobjective Optimization , 2008, Multiobjective Optimization.

[42]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[43]  Sigrún Andradóttir,et al.  A review of simulation optimization techniques , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).