Multi-Model Bayesian Optimization for Simulation-Based Design

We enhance the Bayesian optimization (BO) approach for simulation-based design of engineering systems consisting of multiple interconnected expensive simulation models. The goal is to find the global optimum design with minimal model evaluation costs. A commonly used approach is to treat the whole system as a single expensive model and apply an existing BO algorithm. This approach is inefficient due to the need to evaluate all the component models in each iteration. We propose a multi-model BO approach that dynamically and selectively evaluates one component model per iteration based on the uncertainty quantification of linked emulators (metamodels) and the knowledge gradient of system response as the acquisition function. Building on our basic formulation, we further solve problems with constraints and feedback couplings that often occur in real complex engineering design by penalizing the objective emulator and reformulating the original problem into a decoupled one. The superior efficiency of our approach is demonstrated through solving two analytical problems and the design optimization of a multidisciplinary electronic packaging system.

[1]  Sébastien Le Digabel,et al.  Modeling an Augmented Lagrangian for Blackbox Constrained Optimization , 2014, Technometrics.

[2]  Warren B. Powell,et al.  The Knowledge-Gradient Policy for Correlated Normal Beliefs , 2009, INFORMS J. Comput..

[3]  David H. Wolpert,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996, Neural Computation.

[4]  Victor Picheny,et al.  Adaptive Designs of Experiments for Accurate Approximation of a Target Region , 2010 .

[5]  James O. Berger,et al.  Coupling Computer Models through Linking Their Statistical Emulators , 2018, SIAM/ASA J. Uncertain. Quantification.

[6]  Xiaoping Du,et al.  Efficient Uncertainty Analysis Methods for Multidisciplinary Robust Design , 2002 .

[7]  Ilan Kroo,et al.  Enhanced Collaborative Optimization: Application to an Analytic Test Problem and Aircraft Design , 2008 .

[8]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[9]  Seyede Fatemeh Ghoreishi,et al.  Bayesian Optimization for Efficient Design of Uncertain Coupled Multidisciplinary Systems , 2020, 2020 American Control Conference (ACC).

[10]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[11]  Douglas Allaire,et al.  A Fusion-Based Multi-Information Source Optimization Approach using Knowledge Gradient Policies , 2018 .

[12]  Jonas Mockus,et al.  On Bayesian Methods for Seeking the Extremum , 1974, Optimization Techniques.

[13]  Warren B. Powell,et al.  The Correlated Knowledge Gradient for Simulation Optimization of Continuous Parameters using Gaussian Process Regression , 2011, SIAM J. Optim..

[14]  Peter I. Frazier,et al.  Bayesian Optimization of Composite Functions , 2019, ICML.

[15]  Matthew W. Hoffman,et al.  Predictive Entropy Search for Bayesian Optimization with Unknown Constraints , 2015, ICML.

[16]  Xiaoping Du,et al.  AN EFFICIENT APPROACH TO PROBABILISTIC UNCERTAINTY ANALYSIS IN SIMULATION-BASED MULTIDISCIPLINARY DESIGN , 2000 .

[17]  Victor Picheny,et al.  A Stepwise uncertainty reduction approach to constrained global optimization , 2014, AISTATS.

[18]  Guoqin Shi,et al.  EVALUATION AND IMPLEMENTATION OF MULTIDISCIPLINARY DESIGN OPTIMIZATION STRATEGIES , 2002 .

[19]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[20]  Sankaran Mahadevan,et al.  Multidisciplinary Optimization under Uncertainty Using Bayesian Network , 2016 .

[21]  Matt J. Kusner,et al.  Bayesian Optimization with Inequality Constraints , 2014, ICML.

[22]  Wei Chen,et al.  Reduction of Epistemic Model Uncertainty in Simulation-Based Multidisciplinary Design , 2016 .

[23]  Peter I. Frazier,et al.  The Parallel Knowledge Gradient Method for Batch Bayesian Optimization , 2016, NIPS.

[24]  Donald R. Jones,et al.  Global versus local search in constrained optimization of computer models , 1998 .

[25]  Daniel W. Apley,et al.  Objective-Oriented Sequential Sampling for Simulation Based Robust Design Considering Multiple Sources of Uncertainty , 2013 .

[26]  Guang Yang,et al.  Enhanced Gaussian Process Metamodeling and Collaborative Optimization for Vehicle Suspension Design Optimization , 2017, DAC 2017.

[27]  Agathe Girard,et al.  Gaussian Processes: Prediction at a Noisy Input and Application to Iterative Multiple-Step Ahead Forecasting of Time-Series , 2003, European Summer School on Multi-AgentControl.

[28]  Guang Yang,et al.  Enhanced Collaborative Optimization Using Alternating Direction Method of Multipliers , 2018 .

[29]  I. Sobol.  On the distribution of points in a cube and the approximate evaluation of integrals , 1967 .

[30]  G. B. Olson,et al.  Computational Design of Hierarchically Structured Materials , 1997 .

[31]  Jaroslaw Sobieszczanski-Sobieski,et al.  Multidisciplinary design optimisation - some formal methods, framework requirements, and application to vehicle design , 2001 .

[32]  Wei Chen,et al.  Global Emulation Through Normative Decision Making and Thrifty Adaptive Batch Sampling , 2019, DAC 2019.

[33]  John E. Renaud,et al.  Response surface based, concurrent subspace optimization for multidisciplinary system design , 1996 .

[34]  S. Chen,et al.  EVALUATION OF THREE DECOMPOSITION MDO ALGORITHMS , 2002 .

[35]  Guilherme Ottoni,et al.  Constrained Bayesian Optimization with Noisy Experiments , 2017, Bayesian Analysis.

[36]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[37]  Michael A. Gelbart,et al.  Constrained Bayesian Optimization and Applications , 2015 .

[38]  Joaquim R. R. A. Martins,et al.  Multidisciplinary design optimization: A survey of architectures , 2013 .

[39]  Wei Chen,et al.  A New Variable-Fidelity Optimization Framework Based on Model Fusion and Objective-Oriented Sequential Sampling , 2007, DAC 2007.

[40]  Wei Chen,et al.  A Spatial-Random-Process Based Multidisciplinary System Uncertainty Propagation Approach With Model Uncertainty , 2015 .

[41]  Philipp Hennig,et al.  Entropy Search for Information-Efficient Global Optimization , 2011, J. Mach. Learn. Res..

[42]  Harold J. Kushner,et al.  A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise , 1964 .

[43]  Sankaran Mahadevan,et al.  Test Resource Allocation in Hierarchical Systems Using Bayesian Networks , 2013 .

[44]  Sang-Hoon Lee,et al.  A comparative study of uncertainty propagation methods for black-box-type problems , 2008 .

[45]  Jasper Snoek,et al.  Bayesian Optimization with Unknown Constraints , 2014, UAI.

[46]  A. Sudjianto,et al.  An Efficient Algorithm for Constructing Optimal Design of Computer Experiments , 2005, DAC 2003.