Derivative‐free optimization for expensive constrained problems using a novel expected improvement objective function

In this work, an algorithm for the optimization of costly constrained systems is introduced. The proposed method combines advantages of global- and local-search algorithms with new concepts of feasibility space mapping, within a framework that aims to find global solutions with minimum sampling. A global search is initially performed, during which kriging surrogate models of the objective and the feasible region are developed. A novel search criterion for locating feasibility boundaries is introduced, which does not require any assumptions regarding the convexity and nonlinearity of the feasible space. Finally, local search is performed starting from multiple locations identified by clustering of previously obtained samples. The performance of the proposed approach is evaluated through both benchmark examples and a case study from the pharmaceutical industry. A comparison of the method with commercially available software reveals that the proposed method has a competitive performance in terms of sampling requirements and quality of solution. © 2014 American Institute of Chemical Engineers AIChE J, 60: 2462–2474, 2014

[1]  Charles Audet,et al.  OrthoMADS: A Deterministic MADS Instance with Orthogonal Directions , 2008, SIAM J. Optim..

[2]  Mengqiu Wang,et al.  A hybrid multi-level optimization approach for the dynamic synthesis/design and operation/control un , 2011 .

[3]  C. Floudas,et al.  Active constraint strategy for flexibility analysis in chemical processes , 1987 .

[4]  Deng Huang,et al.  Experimental planning and sequential kriging optimization using variable fidelity data , 2005 .

[5]  I. Grossmann,et al.  An algorithm for the use of surrogate models in modular flowsheet optimization , 2008 .

[6]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[7]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[8]  Kwang-Yong Kim,et al.  Enhanced multi-objective optimization of a microchannel heat sink through evolutionary algorithm coupled with multiple surrogate models , 2010 .

[9]  Christodoulos A. Floudas,et al.  Global Optimization in Design under Uncertainty: Feasibility Test and Flexibility Index Problems , 2001 .

[10]  M. Sasena,et al.  Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization , 2002 .

[11]  Tao Chen,et al.  Efficient meta-modelling of complex process simulations with time-space-dependent outputs , 2011, Comput. Chem. Eng..

[12]  Marianthi G. Ierapetritou,et al.  Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method , 2012, Comput. Chem. Eng..

[13]  Marianthi G. Ierapetritou,et al.  A centroid-based sampling strategy for kriging global modeling and optimization , 2009 .

[14]  Christof Vömel,et al.  ScaLAPACK's MRRR algorithm , 2010, TOMS.

[15]  Selden B. Crary,et al.  Design of Computer Experiments for Metamodel Generation , 2002 .

[16]  Tao Yu,et al.  Reliable multi-objective optimization of high-speed WEDM process based on Gaussian process regression , 2008 .

[17]  Christine A. Shoemaker,et al.  A quasi-multistart framework for global optimization of expensive functions using response surface models , 2013, J. Glob. Optim..

[18]  Scott M. LaLonde,et al.  Understanding the effects of polymer extrusion filter layering configurations using simulation-based optimization , 2010 .

[19]  Christos T. Maravelias,et al.  Surrogate‐based superstructure optimization framework , 2011 .

[20]  Marianthi G. Ierapetritou,et al.  Surrogate-Based Optimization of Expensive Flowsheet Modeling for Continuous Pharmaceutical Manufacturing , 2013, Journal of Pharmaceutical Innovation.

[21]  Gintaras V. Reklaitis,et al.  Simulation-based optimization with surrogate models - Application to supply chain management , 2005, Comput. Chem. Eng..

[22]  Eric Walter,et al.  Global optimization of expensive-to-evaluate functions: an empirical comparison of two sampling criteria , 2009, J. Glob. Optim..

[23]  Spandan Maiti,et al.  Computationally efficient black-box modeling for feasibility analysis , 2010, Comput. Chem. Eng..

[24]  Sébastien Le Digabel,et al.  Use of quadratic models with mesh-adaptive direct search for constrained black box optimization , 2011, Optim. Methods Softw..

[25]  Nikolaos V. Sahinidis,et al.  Derivative-free optimization: a review of algorithms and comparison of software implementations , 2013, J. Glob. Optim..

[26]  Richard C. Baliban,et al.  Modeling, Simulation, and Optimization of Postcombustion CO2 Capture for Variable Feed Concentration and Flow Rate. 2. Pressure Swing Adsorption and Vacuum Swing Adsorption Processes , 2012 .

[27]  Marianthi G. Ierapetritou,et al.  An integrated approach for dynamic flowsheet modeling and sensitivity analysis of a continuous tablet manufacturing process , 2012, Comput. Chem. Eng..

[28]  N. Zheng,et al.  Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models , 2006, J. Glob. Optim..

[29]  Rommel G. Regis,et al.  Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions , 2011, Comput. Oper. Res..

[30]  Marianthi G. Ierapetritou,et al.  New Approach for Quantifying Process Feasibility: Convex and 1-D Quasi-Convex Regions , 2001 .

[31]  Charles Audet,et al.  Convergence of Mesh Adaptive Direct Search to Second-Order Stationary Points , 2006, SIAM J. Optim..

[32]  J. Yin,et al.  Kriging metamodel with modified nugget-effect: The heteroscedastic variance case , 2011, Comput. Ind. Eng..

[33]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[34]  James M. Parr,et al.  Infill sampling criteria for surrogate-based optimization with constraint handling , 2012 .

[35]  Ignacio E. Grossmann,et al.  Optimal process design under uncertainty , 1983 .

[36]  Charles Audet,et al.  A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation , 2010, J. Comput. Phys..

[37]  Marianthi G. Ierapetritou,et al.  A kriging based method for the solution of mixed-integer nonlinear programs containing black-box functions , 2009, J. Glob. Optim..

[38]  Charles Audet,et al.  A MADS Algorithm with a Progressive Barrier for Derivative-Free Nonlinear Programming , 2007 .

[39]  Marianthi G. Ierapetritou,et al.  Feasibility Evaluation of Nonconvex Systems Using Shape Reconstruction Techniques , 2005 .

[40]  M. Patriksson,et al.  A method for simulation based optimization using radial basis functions , 2010 .

[41]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[42]  M. Ierapetritou,et al.  A kriging method for the solution of nonlinear programs with black‐box functions , 2007 .

[43]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[44]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[45]  Marianthi G. Ierapetritou,et al.  Framework for evaluating the feasibility/operability of nonconvex processes , 2003 .

[46]  Sébastien Le Digabel,et al.  Algorithm xxx : NOMAD : Nonlinear Optimization with the MADS algorithm , 2010 .

[47]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[48]  Christine A. Shoemaker,et al.  SO-MI: A surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems , 2013, Comput. Oper. Res..

[49]  Marianthi G. Ierapetritou,et al.  Design Optimization under Parameter Uncertainty for General Black-Box Models , 2002 .

[50]  Charles Audet,et al.  A Pattern Search Filter Method for Nonlinear Programming without Derivatives , 2001, SIAM J. Optim..

[51]  Silvana M. B. Afonso,et al.  A concurrent efficient global optimization algorithm applied to polymer injection strategies , 2010 .

[52]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[53]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[54]  Charles Audet,et al.  Nonsmooth optimization through Mesh Adaptive Direct Search and Variable Neighborhood Search , 2006, J. Glob. Optim..

[55]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[56]  Noel A. C. Cressie,et al.  Statistics for Spatial Data: Cressie/Statistics , 1993 .

[57]  Marianthi G. Ierapetritou,et al.  A Kriging-Based Approach to MINLP Containing Black-Box Models and Noise , 2008 .