UNIPOPT: Univariate projection-based optimization without derivatives

Abstract A derivative-free optimization framework UNIPOPT is developed to solve multi-dimensional box-constrained black-box problems. The method is based on projecting the objective function onto a univariate space, which leads to a point-to-set map. The lower envelope of the map contains the global minimum. A sensitivity theorem is employed to predict the points on the lower envelope and a trust-region based algorithm is used to correct the predictions. A model-based algorithm is used to optimize the univariate lower envelope such that its minimum corresponds to that of the original problem. We provide theoretical results related to the prediction and correction steps and prove the convergence of the overall algorithm. UNIPOPT is applied to an extensive suite of test problems comprising of both convex nonsmooth and nonconvex smooth black-box problems, and its performance is compared to the existing model-based approaches. The algorithm is able to locate global solutions for more test problems compared to the other solvers.

[1]  Alberto Bemporad,et al.  Dynamic programming for constrained optimal control of discrete-time linear hybrid systems , 2005, Autom..

[2]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[3]  Sigurd Skogestad,et al.  Surrogate model generation using self-optimizing variables , 2018, Comput. Chem. Eng..

[4]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[5]  Christodoulos A Floudas,et al.  Cost-effective CO2 capture based on in silico screening of zeolites and process optimization. , 2013, Physical chemistry chemical physics : PCCP.

[6]  Atharv Bhosekar,et al.  Advances in surrogate based modeling, feasibility analysis, and optimization: A review , 2018, Comput. Chem. Eng..

[7]  Arthur M. Geoffrion,et al.  Primal Resource-Directive Approaches for Optimizing Nonlinear Decomposable Systems , 1970, Oper. Res..

[8]  Christos T. Maravelias,et al.  Surrogate‐based superstructure optimization framework , 2011 .

[9]  A. Fiacco,et al.  Convexity and concavity properties of the optimal value function in parametric nonlinear programming , 1983 .

[10]  Anthony V. Fiacco,et al.  Introduction to Sensitivity and Stability Analysis in Nonlinear Programming , 2012 .

[11]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[12]  Nikolaos V. Sahinidis,et al.  A polyhedral branch-and-cut approach to global optimization , 2005, Math. Program..

[13]  Iftekhar A. Karimi,et al.  Heating Value Reduction of LNG (Liquefied Natural Gas) by Recovering Heavy Hydrocarbons: Technoeconomic Analyses Using Simulation-Based Optimization , 2018 .

[14]  Ishan Bajaj,et al.  Integrated Carbon Capture and Conversion To Produce Syngas: Novel Process Design, Intensification, and Optimization , 2017 .

[15]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[16]  A. Shapiro.  Sensitivity analysis of nonlinear programs and differentiability properties of metric projections , 1988 .

[17]  CHARLES AUDET,et al.  Finding Optimal Algorithmic Parameters Using Derivative-Free Optimization , 2006, SIAM J. Optim..

[18]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[19]  Nikolaos V. Sahinidis,et al.  Derivative-free optimization: a review of algorithms and comparison of software implementations , 2013, J. Glob. Optim..

[20]  Linus Schrage,et al.  The global solver in the LINDO API , 2009, Optim. Methods Softw..

[21]  Christodoulos A. Floudas,et al.  Discovery of novel zeolites for natural gas purification through combined material screening and process optimization , 2014 .

[22]  Christodoulos A. Floudas,et al.  Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO , 2016, Eur. J. Oper. Res..

[23]  M. Ierapetritou,et al.  A novel feasibility analysis method for black‐box processes using a radial basis function adaptive sampling approach , 2017 .

[24]  Lorenz T. Biegler,et al.  Advanced trust region optimization strategies for glass box/ black box models , 2018, AIChE Journal.

[25]  Christodoulos A. Floudas,et al.  Optimization of black-box problems using Smolyak grids and polynomial approximations , 2018, J. Glob. Optim..

[26]  Philippe L. Toint,et al.  A derivative-free trust-funnel method for equality-constrained nonlinear optimization , 2014, Computational Optimization and Applications.

[27]  Tatyana Plaksina,et al.  On improving the hydrogen and methanol production using an auto-thermal double-membrane reactor: Model prediction and optimisation , 2018, Comput. Chem. Eng..

[28]  Christodoulos A. Floudas,et al.  A multi-scale framework for CO2 capture, utilization, and sequestration: CCUS and CCU , 2015, Comput. Chem. Eng..

[29]  Lorenz T. Biegler,et al.  A trust region filter method for glass box/black box optimization , 2016 .

[30]  Shu-Kai S. Fan,et al.  A hybrid simplex search and particle swarm optimization for unconstrained optimization , 2007, Eur. J. Oper. Res..

[31]  Mohd Shariq Khan,et al.  Design optimization of single mixed refrigerant natural gas liquefaction process using the particle swarm paradigm with nonlinear constraints , 2013 .

[32]  Ishan Bajaj,et al.  Simulation and optimization of reforming reactors for carbon dioxide utilization using both rigorous and reduced models , 2018 .

[33]  Arnold Neumaier,et al.  Global Optimization by Multilevel Coordinate Search , 1999, J. Glob. Optim..

[34]  Iftekhar A. Karimi,et al.  LEAPS2: Learning based Evolutionary Assistive Paradigm for Surrogate Selection , 2018, Comput. Chem. Eng..

[35]  Charles Audet,et al.  Derivative-Free and Blackbox Optimization , 2017 .

[36]  Christodoulos A. Floudas,et al.  ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems , 2017, Optim. Lett..

[37]  M. J. D. Powell,et al.  On the convergence of trust region algorithms for unconstrained minimization without derivatives , 2012, Comput. Optim. Appl..

[38]  R. Meyer.  The Validity of a Family of Optimization Methods , 1970 .

[39]  Arnold Neumaier,et al.  SNOBFIT -- Stable Noisy Optimization by Branch and Fit , 2008, TOMS.

[40]  Sébastien Le Digabel,et al.  Algorithm xxx : NOMAD : Nonlinear Optimization with the MADS algorithm , 2010 .

[41]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[42]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[43]  Tibor Csendes,et al.  Noname manuscript No. (will be inserted by the editor) The GLOBAL Optimization Method Revisited , 2022 .

[44]  Ishan Bajaj,et al.  A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point , 2017, Comput. Chem. Eng..

[45]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[46]  Tamara G. Kolda,et al.  Optimizing an Empirical Scoring Function for Transmembrane Protein Structure Determination , 2004, INFORMS J. Comput..

[47]  E. Pistikopoulos,et al.  A multiparametric programming approach for mixed-integer quadratic engineering problems , 2002 .

[48]  R. Regis.  Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points , 2014 .

[49]  G. Dantzig,et al.  On the continuity of the minimum set of a continuous function , 1967 .

[50]  Nikolaos Ploskas,et al.  Tuning BARON using derivative-free optimization algorithms , 2019, J. Glob. Optim..

[51]  Jerzy Kyparisis,et al.  Sensitivity Analysis for Nonlinear Programs and Variational Inequalities with Nonunique Multipliers , 1990, Math. Oper. Res..

[52]  Alison L. Marsden,et al.  A computational framework for derivative-free optimization of cardiovascular geometries , 2008 .

[53]  Lester Ingber,et al.  Adaptive Simulated Annealing , 2012 .

[54]  W. Hogan.  Point-to-Set Maps in Mathematical Programming , 1973 .

[55]  Selen Cremaschi,et al.  An algorithm to determine sample sizes for optimization with artificial neural networks , 2013 .

[56]  Floyd J. Gould,et al.  Stability in Nonlinear Programming , 1970, Oper. Res..

[57]  Robert L. Smith,et al.  Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions , 1984, Oper. Res..

[58]  A. Auslender,et al.  First and second order sensitivity analysis of nonlinear programs under directional constraint qualification conditions , 1990 .

[59]  Iftekhar A. Karimi,et al.  Design of computer experiments: A review , 2017, Comput. Chem. Eng..

[60]  Luís N. Vicente,et al.  A particle swarm pattern search method for bound constrained global optimization , 2007, J. Glob. Optim..

[61]  M. Powell.  The BOBYQA algorithm for bound constrained optimization without derivatives , 2009 .

[62]  Anh Phong Tran,et al.  On the estimation of high-dimensional surrogate models of steady-state of plant-wide processes characteristics , 2018, Comput. Chem. Eng..

[63]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[64]  Tatsiana Levina,et al.  Dynamic Pricing with Online Learning and Strategic Consumers: An Application of the Aggregating Algorithm , 2009, Oper. Res..

[65]  E. W. Karas,et al.  A globally convergent trust-region algorithm for unconstrained derivative-free optimization , 2015 .

[66]  Alberto Bemporad,et al.  The explicit linear quadratic regulator for constrained systems , 2003, Autom..

[67]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[68]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[69]  I. Grossmann,et al.  An algorithm for the use of surrogate models in modular flowsheet optimization , 2008 .

[70]  Anthony V. Fiacco,et al.  Sensitivity analysis for nonlinear programming using penalty methods , 1976, Math. Program..

[71]  Bernhard Gollan,et al.  On The Marginal Function in Nonlinear Programming , 1984, Math. Oper. Res..

[72]  Luís N. Vicente,et al.  Using Sampling and Simplex Derivatives in Pattern Search Methods , 2007, SIAM J. Optim..

[73]  Christodoulos A. Floudas,et al.  Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption , 2017, J. Glob. Optim..

[74]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[75]  Michael S. Eldred,et al.  DAKOTA : a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. Version 5.0, user's reference manual. , 2010 .

[76]  Christine A. Shoemaker,et al.  Comparison of Optimization Methods for Ground-Water Bioremediation , 1999 .

[77]  K. Jittorntrum.  Solution point differentiability without strict complementarity in nonlinear programming , 1984 .

[78]  Efstratios N. Pistikopoulos,et al.  Optimal design of energy systems using constrained grey-box multi-objective optimization , 2018, Comput. Chem. Eng..

[79]  Ishan Bajaj,et al.  Optimal synthesis of periodic sorption enhanced reaction processes with application to hydrogen production , 2018, Comput. Chem. Eng..

[80]  Thomas A. Adams,et al.  A novel polygeneration process to co-produce ethylene and electricity from shale gas with zero CO2 emissions via methane oxidative coupling , 2015 .

[81]  Nikolaos V. Sahinidis,et al.  A combined first-principles and data-driven approach to model building , 2015, Comput. Chem. Eng..

[82]  C. T. Kelley,et al.  An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima , 1995, SIAM J. Optim..

[83]  Jacques Gauvin,et al.  Directional Behaviour of Optimal Solutions in Nonlinear Mathematical Programming , 1988, Math. Oper. Res..

[84]  Ishan Bajaj,et al.  Optimal Methanol Production via Sorption-Enhanced Reaction Process , 2018, Industrial & Engineering Chemistry Research.

[85]  Robert L. Smith,et al.  Hit-and-Run Algorithms for Generating Multivariate Distributions , 1993, Math. Oper. Res..

[86]  Simon P. Wilson,et al.  Using DIRECT to Solve an Aircraft Routing Problem , 2002, Comput. Optim. Appl..