A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point

Abstract This paper presents an algorithm for constrained black-box and grey-box optimization. It is based on surrogate models developed using input-output data in a trust-region framework. Unlike many current methods, the proposed approach does not require feasible initial point and can handle hard constraints via a novel optimization-based constrained sampling scheme. A two-phase strategy is employed, where the first phase involves finding feasible point through minimizing a smooth constraint violation function (feasibility phase). The second phase improves the objective in the feasible region using the solution of the feasibility phase as starting point (optimization phase). The method is applied to solve 92 test problems and the performance is compared with established derivative-free solvers. The two-phase algorithm outperforms these solvers in terms of number of problems solved and number of samples used. We also apply the algorithm to solve a chemical process design problem involving highly-coupled, nonlinear algebraic and partial differential equations.

[1]  C. T. Kelley,et al.  An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima , 1995, SIAM J. Optim..

[2]  Xin-She Yang,et al.  Metaheuristics in water, geotechnical and transport engineering , 2012 .

[3]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[4]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[5]  Christodoulos A Floudas,et al.  Cost-effective CO2 capture based on in silico screening of zeolites and process optimization. , 2013, Physical chemistry chemical physics : PCCP.

[6]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[7]  Charles Audet,et al.  A MADS Algorithm with a Progressive Barrier for Derivative-Free Nonlinear Programming , 2007 .

[8]  S. Nilchan,et al.  On the Optimisation of Periodic Adsorption Processes , 1998 .

[9]  Iftekhar A. Karimi,et al.  Design of computer experiments: A review , 2017, Comput. Chem. Eng..

[10]  M. Hasan,et al.  Effective Sampling, Modeling and Optimization of Constrained Black-box Problems , 2016 .

[11]  Charles Audet,et al.  A Pattern Search Filter Method for Nonlinear Programming without Derivatives , 2001, SIAM J. Optim..

[12]  Nicholas I. M. Gould,et al.  Nonlinear programming without a penalty function or a filter , 2010, Math. Program..

[13]  Yurii Nesterov,et al.  Random Gradient-Free Minimization of Convex Functions , 2015, Foundations of Computational Mathematics.

[14]  F. J. Hickernell.  Lattice rules: how well do they measure up? in random and quasi-random point sets , 1998 .

[15]  Luís N. Vicente,et al.  A particle swarm pattern search method for bound constrained global optimization , 2007, J. Glob. Optim..

[16]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[17]  M. Powell.  A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[18]  Nikolaos V. Sahinidis,et al.  Derivative-free optimization: a review of algorithms and comparison of software implementations , 2013, J. Glob. Optim..

[19]  Stephen E. Zitney,et al.  A Superstructure-Based Optimal Synthesis of PSA Cycles for Post-Combustion CO2 Capture , 2009 .

[20]  Lorenz T. Biegler,et al.  A trust region filter method for glass box/black box optimization , 2016 .

[21]  Arnold Neumaier,et al.  SNOBFIT -- Stable Noisy Optimization by Branch and Fit , 2008, TOMS.

[22]  Nélida E. Echebest,et al.  Inexact Restoration method for nonlinear optimization without derivatives , 2015, J. Comput. Appl. Math..

[23]  Christodoulos A. Floudas,et al.  Discovery of novel zeolites for natural gas purification through combined material screening and process optimization , 2014 .

[24]  Marianthi G. Ierapetritou,et al.  Feasibility and flexibility analysis of black-box processes Part 1: Surrogate-based feasibility analysis , 2015 .

[25]  Atharv Bhosekar,et al.  Advances in surrogate based modeling, feasibility analysis, and optimization: A review , 2018, Comput. Chem. Eng..

[26]  Christos T. Maravelias,et al.  Surrogate‐based superstructure optimization framework , 2011 .

[27]  Leifur Leifsson,et al.  Surrogate-Based Methods , 2011, Computational Optimization, Methods and Algorithms.

[28]  J. M. Martínez,et al.  Derivative-free methods for nonlinear programming with general lower-level constraints , 2011 .

[29]  Ishan Bajaj,et al.  Integrated Carbon Capture and Conversion To Produce Syngas: Novel Process Design, Intensification, and Optimization , 2017 .

[30]  Stefano Lucidi,et al.  A Derivative-Free Algorithm for Inequality Constrained Nonlinear Programming via Smoothing of an linfty Penalty Function , 2009, SIAM J. Optim..

[31]  Sébastien Le Digabel,et al.  Algorithm xxx : NOMAD : Nonlinear Optimization with the MADS algorithm , 2010 .

[32]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[33]  Galo A. C. Le Roux,et al.  Improvements in surrogate models for process synthesis. Application to water network system design , 2013, Comput. Chem. Eng..

[34]  Herbert Hamers,et al.  Constrained optimization involving expensive function evaluations: A sequential approach , 2005, Eur. J. Oper. Res..

[35]  Elizabeth W. Karas,et al.  A trust-region derivative-free algorithm for constrained optimization , 2015, Optim. Methods Softw..

[36]  Katya Scheinberg,et al.  Geometry of interpolation sets in derivative free optimization , 2007, Math. Program..

[37]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[38]  Charles Audet,et al.  Spent potliner treatment process optimization using a MADS algorithm , 2005 .

[39]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[40]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[41]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[42]  Fernando J. Muzzio,et al.  Predictive Modeling for Pharmaceutical Processes Using Kriging and Response Surface , 2009, Journal of Pharmaceutical Innovation.

[43]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .

[44]  Philippe L. Toint,et al.  A derivative-free trust-funnel method for equality-constrained nonlinear optimization , 2014, Computational Optimization and Applications.

[45]  Stefan M. Wild,et al.  CONORBIT: constrained optimization by radial basis function interpolation in trust regions† , 2017, Optim. Methods Softw..

[46]  Fabiano A.N. Fernandes,et al.  Optimization of Fischer‐Tropsch Synthesis Using Neural Networks , 2006 .

[47]  V. Torczon,et al.  Direct search methods: then and now , 2000 .

[48]  Christodoulos A. Floudas,et al.  ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems , 2017, Optim. Lett..

[49]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[50]  T. Ray,et al.  A framework for design optimization using surrogates , 2005 .

[51]  Charles Audet,et al.  OrthoMADS: A Deterministic MADS Instance with Orthogonal Directions , 2008, SIAM J. Optim..

[52]  Marco Sciandrone,et al.  Sequential Penalty Derivative-Free Methods for Nonlinear Constrained Optimization , 2010, SIAM J. Optim..

[53]  L. Biegler,et al.  Optimization strategies for simulated moving bed and PowerFeed processes , 2006 .

[54]  I. Grossmann,et al.  An algorithm for the use of surrogate models in modular flowsheet optimization , 2008 .

[55]  Luís N. Vicente,et al.  Using Sampling and Simplex Derivatives in Pattern Search Methods , 2007, SIAM J. Optim..

[56]  Yaroslav D. Sergeyev,et al.  Lipschitz Global Optimization , 2011 .

[57]  A. I. Stankiewicz,et al.  Process Intensification: Transforming Chemical Engineering , 2000 .

[58]  Christodoulos A. Floudas,et al.  ANTIGONE: Algorithms for coNTinuous / Integer Global Optimization of Nonlinear Equations , 2014, Journal of Global Optimization.

[59]  R. Regis.  Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points , 2014 .

[60]  Matthew J. Realff,et al.  Metamodeling Approach to Optimization of Steady-State Flowsheet Simulations: Model Generation , 2002 .

[61]  M. Rijckaert,et al.  Intelligent modelling in the chemical process industry with neural networks : A case study , 1998 .

[62]  Selen Cremaschi,et al.  Process synthesis of biodiesel production plant using artificial neural networks as the surrogate models , 2012, Comput. Chem. Eng..

[63]  Katya Scheinberg,et al.  Self-Correcting Geometry in Model-Based Algorithms for Derivative-Free Unconstrained Optimization , 2010, SIAM J. Optim..

[64]  Charles Audet,et al.  A progressive barrier derivative-free trust-region algorithm for constrained optimization , 2016, Comput. Optim. Appl..

[65]  Jianping Li,et al.  Systematic process intensification using building blocks , 2017, Comput. Chem. Eng..