Constrained optimization of black-box stochastic systems using a novel feasibility enhanced Kriging-based method

Abstract Stochastically constrained simulation optimization problems are challenging because the inherent noise terms to a black-box system lead to the need of considering the uncertainty at both the objective and the constraint function. To address this problem, we propose a Kriging-based optimization framework, which uses stochastic Kriging models to approximate the objective and the constraint functions and an adaptive sampling approach to sequentially search for the next point that is promising to have a better objective. The main contribution of this work is to incorporate a feasibility-enhanced term to the infill sampling criterion, which improves the Kriging-based algorithm's capabilities of returning a truly feasible near-optimal solution for stochastic systems. The efficacy of the Kriging-based algorithm is demonstrated with eight benchmark problems and a case study from the pharmaceutical manufacturing domain.

[1]  Susana Gómez,et al.  The tunnelling method for solving the constrained global optimization problem with several non-connected feasible regions , 1982 .

[2]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[3]  Antanas Zilinskas,et al.  Stochastic Global Optimization: A Review on the Occasion of 25 Years of Informatica , 2016, Informatica.

[4]  Scott Kuindersma,et al.  Variable risk control via stochastic optimization , 2013, Int. J. Robotics Res..

[5]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[6]  Kenneth J. Klassen,et al.  Improving Performance in Outpatient Appointment Services with a Simulation Optimization Approach , 2009 .

[7]  Kuo-Hao Chang,et al.  Quantile-Based Simulation Optimization With Inequality Constraints: Methodology and Applications , 2016, IEEE Transactions on Automation Science and Engineering.

[8]  Min Xie,et al.  A systematic comparison of metamodeling techniques for simulation optimization in Decision Support Systems , 2010, Appl. Soft Comput..

[9]  Jing J. Liang,et al.  Problem Deflnitions and Evaluation Criteria for the CEC 2006 Special Session on Constrained Real-Parameter Optimization , 2006 .

[10]  Matthijs Langelaar,et al.  Expected improvement based infill sampling for global robust optimization of constrained problems , 2017 .

[11]  Jack P. C. Kleijnen,et al.  Constrained Optimization in Simulation: A Novel Approach , 2008 .

[12]  Loo Hay Lee,et al.  Simulation optimization via kriging: a sequential search using expected improvement with computing budget constraints , 2013 .

[13]  Jack P. C. Kleijnen,et al.  Kriging Metamodeling in Simulation: A Review , 2007, Eur. J. Oper. Res..

[14]  Sujin Kim,et al.  The mathematics of continuous-variable simulation optimization , 2008, 2008 Winter Simulation Conference.

[15]  Nikolaos V. Sahinidis,et al.  Simulation optimization: a review of algorithms and applications , 2014, 4OR.

[16]  Fernando J. Muzzio,et al.  Feedrate deviations caused by hopper refill of loss-in-weight feeders , 2015 .

[17]  Zilong Wang,et al.  Process analysis and optimization of continuous pharmaceutical manufacturing using flowsheet models , 2017, Comput. Chem. Eng..

[18]  Xi Chen,et al.  Sequential design strategies for mean response surface metamodeling via stochastic kriging with adaptive exploration and exploitation , 2017, Eur. J. Oper. Res..

[19]  Zilong Wang,et al.  Surrogate-based feasibility analysis for black-box stochastic simulations with heteroscedastic noise , 2018, J. Glob. Optim..

[20]  James M. Parr,et al.  Infill sampling criteria for surrogate-based optimization with constraint handling , 2012 .

[21]  Efstratios N. Pistikopoulos,et al.  Optimal design of energy systems using constrained grey-box multi-objective optimization , 2018, Comput. Chem. Eng..

[22]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[23]  A. Forrester,et al.  Design and analysis of 'noisy' computer experiments , 2006 .

[24]  T. Lacksonen.  Empirical comparison of search algorithms for discrete event simulation , 2001 .

[25]  Sébastien Le Digabel,et al.  Algorithm xxx : NOMAD : Nonlinear Optimization with the MADS algorithm , 2010 .

[26]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[27]  Jack P. C. Kleijnen,et al.  Simulation-optimization via Kriging and bootstrapping: a survey , 2014, J. Simulation.

[28]  Xin Yao,et al.  Search biases in constrained evolutionary optimization , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[29]  Hashem Mahlooji,et al.  An artificial neural network meta-model for constrained simulation optimization , 2014, J. Oper. Res. Soc..

[30]  Shalabh Bhatnagar,et al.  Stochastic approximation algorithms for constrained optimization via simulation , 2011, TOMC.

[31]  D. Ginsbourger,et al.  A benchmark of kriging-based infill criteria for noisy optimization , 2013, Structural and Multidisciplinary Optimization.

[32]  Ignacio E. Grossmann,et al.  Evolution of concepts and models for quantifying resiliency and flexibility of chemical processes , 2014, Comput. Chem. Eng..

[33]  Kalyanmoy Deb,et al.  Constraint handling in efficient global optimization , 2017, GECCO.

[34]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[35]  Christodoulos A. Floudas,et al.  Global optimization of grey-box computational systems using surrogate functions and application to highly constrained oil-field operations , 2018, Comput. Chem. Eng..

[36]  Tapabrata Ray,et al.  A study on the effectiveness of constraint handling schemes within Efficient Global Optimization framework , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[37]  Andy J. Keane,et al.  Enhancing infill sampling criteria for surrogate-based constrained optimization , 2012, J. Comput. Methods Sci. Eng..

[38]  Jack P. C. Kleijnen,et al.  Constrained optimization in expensive simulation: Novel approach , 2010, Eur. J. Oper. Res..

[39]  Eric Walter,et al.  An informational approach to the global optimization of expensive-to-evaluate functions , 2006, J. Glob. Optim..

[40]  Gül Gürkan,et al.  Response surface methodology with stochastic constraints for expensive simulation , 2009, J. Oper. Res. Soc..

[41]  Sujin Kim,et al.  Convergence properties of direct search methods for stochastic optimization , 2010, Proceedings of the 2010 Winter Simulation Conference.

[42]  Atharv Bhosekar,et al.  Advances in surrogate based modeling, feasibility analysis, and optimization: A review , 2018, Comput. Chem. Eng..

[43]  A. ilinskas.  Axiomatic characterization of a global optimization algorithm and investigation of its search strategy , 1985 .

[44]  Michael C. Fu,et al.  Optimization for Simulation: Theory vs. Practice , 2002 .

[45]  Fernando J. Muzzio,et al.  Method for characterization of loss-in-weight feeder equipment , 2012 .

[46]  M. Ierapetritou,et al.  A Novel Surrogate-Based Optimization Method for Black-Box Simulation with Heteroscedastic Noise , 2017 .

[47]  Zilong Wang,et al.  Surrogate-based Optimization for Pharmaceutical Manufacturing Processes , 2017 .

[48]  Marianthi G. Ierapetritou,et al.  Derivative‐free optimization for expensive constrained problems using a novel expected improvement objective function , 2014 .

[49]  K. Preston White,et al.  Stochastic approximation with simulated annealing as an approach to global discrete-event simulation optimization , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[50]  Christodoulos A. Floudas,et al.  ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems , 2017, Optim. Lett..

[51]  Victor Picheny,et al.  Quantile-Based Optimization of Noisy Computer Experiments With Tunable Precision , 2013, Technometrics.

[52]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[53]  Jack P. C. Kleijnen,et al.  Efficient global optimisation for black-box simulation via sequential intrinsic Kriging , 2018, J. Oper. Res. Soc..

[54]  Jack P. C. Kleijnen,et al.  Stochastic Intrinsic Kriging for Simulation Metamodelling , 2015 .

[55]  Ihsan Sabuncuoglu,et al.  Simulation optimization: A comprehensive review on theory and applications , 2004 .

[56]  Barry L. Nelson,et al.  Stochastic kriging for simulation metamodeling , 2008, 2008 Winter Simulation Conference.

[57]  M. Sasena,et al.  Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization , 2002 .

[58]  Victor Picheny,et al.  Comparison of Kriging-based algorithms for simulation optimization with heterogeneous noise , 2017, Eur. J. Oper. Res..

[59]  Eric Walter,et al.  Global optimization based on noisy evaluations: An empirical study of two statistical approaches , 2008 .

[60]  Christodoulos A. Floudas,et al.  Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption , 2017, J. Glob. Optim..

[61]  N. Zheng,et al.  Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models , 2006, J. Glob. Optim..

[62]  Kalyanmoy Deb,et al.  Constrained Efficient Global Optimization for Pultrusion Process , 2015 .

[63]  Christodoulos A. Floudas,et al.  Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO , 2016, Eur. J. Oper. Res..

[64]  Inneke Van Nieuwenhuyse,et al.  Simulation optimization in inventory replenishment: a classification , 2015 .

[65]  Michael James Sasena,et al.  Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. , 2002 .