Stochastic Multi-objective Optimization on a Budget: Application to multi-pass wire drawing with quantified uncertainties

Design optimization of engineering systems with multiple competing objectives is a painstakingly tedious process especially when the objective functions are expensive-to-evaluate computer codes with parametric uncertainties. The effectiveness of the state-of-the-art techniques is greatly diminished because they require a large number of objective evaluations, which makes them impractical for problems of the above kind. Bayesian global optimization (BGO), has managed to deal with these challenges in solving single-objective optimization problems and has recently been extended to multi-objective optimization (MOO). BGO models the objectives via probabilistic surrogates and uses the epistemic uncertainty to define an information acquisition function (IAF) that quantifies the merit of evaluating the objective at new designs. This iterative data acquisition process continues until a stopping criterion is met. The most commonly used IAF for MOO is the expected improvement over the dominated hypervolume (EIHV) which in its original form is unable to deal with parametric uncertainties or measurement noise. In this work, we provide a systematic reformulation of EIHV to deal with stochastic MOO problems. The primary contribution of this paper lies in being able to filter out the noise and reformulate the EIHV without having to observe or estimate the stochastic parameters. An addendum of the probabilistic nature of our methodology is that it enables us to characterize our confidence about the predicted Pareto front. We verify and validate the proposed methodology by applying it to synthetic test problems with known solutions. We demonstrate our approach on an industrial problem of die pass design for a steel wire drawing process.

[1]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[2]  David Ginsbourger,et al.  A Multi-points Criterion for Deterministic Parallel Global Optimization based on Kriging , 2007 .

[3]  J. Mockus Bayesian Approach to Global Optimization: Theory and Applications , 1989 .

[4]  Kalyanmoy Deb,et al.  Introduction to Evolutionary Multiobjective Optimization , 2008, Multiobjective Optimization.

[5]  Jonas Mockus,et al.  On Bayesian Methods for Seeking the Extremum , 1974, Optimization Techniques.

[6]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[7]  M. Emmerich,et al.  The computation of the expected improvement in dominated hypervolume of Pareto front approximations , 2008 .

[8]  James M. Parr,et al.  Improvement criteria for constraint handling and multiobjective optimization , 2013 .

[9]  Shigeru Obayashi,et al.  Updating Kriging Surrogate Models Based on the Hypervolume Indicator in Multi-Objective Optimization , 2013 .

[10]  R. Baierlein Probability Theory: The Logic of Science , 2004 .

[11]  David Ginsbourger,et al.  Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations , 2015, Eur. J. Oper. Res..

[12]  J. Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[13]  Piyush Pandita,et al.  Extending Expected Improvement for High-Dimensional Stochastic Optimization of Expensive Black-Box Functions , 2016, DAC 2016.

[14]  Andy J. Keane,et al.  Statistical Improvement Criteria for Use in Multiobjective Design Optimization , 2006 .

[15]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[16]  Milan Zeleny,et al.  The pros and cons of goal programming , 1981, Comput. Oper. Res..

[17]  W. Marsden I and J , 2012 .

[18]  N. Cressie The origins of kriging , 1990 .

[19]  A. Charnes,et al.  Goal programming and multiple objective optimizations: Part 1 , 1977 .

[20]  Warren B. Powell,et al.  Optimal Learning: Powell/Optimal , 2012 .

[21]  Jasbir S. Arora,et al.  Survey of multi-objective optimization methods for engineering , 2004 .

[22]  Daniel P. Heyman,et al.  Stochastic models in operations research , 1982 .

[23]  N. Zheng,et al.  Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models , 2006, J. Glob. Optim..

[24]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[25]  David Ginsbourger,et al.  Estimating and quantifying uncertainties on level sets using the Vorob'ev expectation and deviation with Gaussian process models , 2013 .

[26]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[27]  Julien Bect,et al.  A Bayesian Approach to Constrained Multi-objective Optimization , 2015, LION.

[28]  I. Molchanov Theory of Random Sets , 2005 .

[29]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[30]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[31]  Howie Choset,et al.  Expensive multiobjective optimization for robotics , 2013, 2013 IEEE International Conference on Robotics and Automation.