Surrogate-based modeling techniques with application to catalytic reforming and isomerization processes

Abstract In this paper, we first briefly survey the main surrogate model building approaches discussed in the literature considering also design of experiments strategies and dimensionality reduction procedures: we mainly focus on sub-set approaches and sampling strategies for constrained regression problems. We delineate a systematic methodology for surrogate modelling in presence of model constraints, such as non-negativity of the model responses. The main contribution of this paper is twofold: from one side we extend the principal component analysis framework to the case of constrained regression problem, from the other we propose a novel methodology which integrates the subset selection and the previous principal component regression procedure. Finally, we apply the two novel algorithms to two fundamental chemical processes in petroleum refinery, namely catalytic reforming and light naphtha isomerization. The numerical results show the comparisons between the two algorithms in terms of computational and accuracy trade-offs.

[1]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[2]  Nikolaos V. Sahinidis,et al.  The ALAMO approach to machine learning , 2017, Comput. Chem. Eng..

[3]  J. Ancheyta-Juárez,et al.  Experimental validation of a kinetic model for naphtha reforming , 2001 .

[4]  T. Simpson,et al.  Analysis of support vector regression for approximation of complex engineering analyses , 2005, DAC 2003.

[5]  Qing-Song Xu,et al.  Support vector machines and its applications in chemistry , 2009 .

[6]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[7]  D. Krige.  A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[8]  Christine A. Shoemaker,et al.  Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems , 2014, J. Glob. Optim..

[9]  J. S. Hunter,et al.  Statistics for Experimenters: Design, Innovation, and Discovery , 2006 .

[10]  Bertrand Iooss,et al.  Latin hypercube sampling with inequality constraints , 2009, 0909.0329.

[11]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[12]  Sébastien Le Digabel,et al.  Algorithm xxx : NOMAD : Nonlinear Optimization with the MADS algorithm , 2010 .

[13]  L. Pronzato.  Minimax and maximin space-filling designs: some properties and methods for construction , 2017 .

[14]  Nikolaos V. Sahinidis,et al.  A combined first-principles and data-driven approach to model building , 2015, Comput. Chem. Eng..

[15]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[16]  Selen Cremaschi,et al.  Adaptive sequential sampling for surrogate model generation with artificial neural networks , 2014, Comput. Chem. Eng..

[17]  Iftekhar A. Karimi,et al.  Smart Sampling Algorithm for Surrogate Model Development , 2017, Comput. Chem. Eng..

[18]  Christodoulos A. Floudas,et al.  ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems , 2017, Optim. Lett..

[19]  L. Haines,et al.  The construction of D- and I-optimal designs for mixture experiments with linear constraints on the components , 2017 .

[20]  Edwin R. van Dam,et al.  Two-Dimensional Minimax Latin Hypercube Designs , 2005, Discret. Appl. Math..

[21]  P. R. Pujadó,et al.  Isomerization in Petroleum Processing , 2015 .

[22]  David C. Miller,et al.  Learning surrogate models for simulation‐based optimization , 2014 .

[23]  Christodoulos A. Floudas,et al.  Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO , 2016, Eur. J. Oper. Res..

[24]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[25]  Felipe A. C. Viana,et al.  A Tutorial on Latin Hypercube Design of Experiments , 2016, Qual. Reliab. Eng. Int..

[26]  Efstratios N. Pistikopoulos,et al.  Optimal design of energy systems using constrained grey-box multi-objective optimization , 2018, Comput. Chem. Eng..

[27]  Alexandre Pagot,et al.  Optimization approaches to the integrated system of catalytic reforming and isomerization processes in petroleum refinery , 2020, Comput. Chem. Eng..

[28]  Trevor Hastie,et al.  Statistical Learning with Sparsity: The Lasso and Generalizations , 2015 .

[29]  Christodoulos A. Floudas,et al.  Global optimization of grey-box computational systems using surrogate functions and application to highly constrained oil-field operations , 2018, Comput. Chem. Eng..

[30]  Christodoulos A. Floudas,et al.  Dimensionality reduction for production optimization using polynomial approximations , 2017, Computational Geosciences.

[31]  Alan J. Miller.  Subset Selection in Regression , 1992 .

[32]  Atharv Bhosekar,et al.  Advances in surrogate based modeling, feasibility analysis, and optimization: A review , 2018, Comput. Chem. Eng..

[33]  N. Strömberg,et al.  Radial basis functions as surrogate models with a priori bias in comparison with a posteriori bias , 2017 .

[34]  G. Matheron.  Principles of geostatistics , 1963 .

[35]  Carlos Guedes Soares,et al.  Adaptive surrogate model with active refinement combining Kriging and a trust region method , 2017, Reliab. Eng. Syst. Saf..

[36]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[37]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[38]  Kai Sundmacher,et al.  Overview of Surrogate Modeling in Chemical Process Engineering , 2019, Chemie Ingenieur Technik.

[39]  J. Morlier,et al.  Improving kriging surrogates of high-dimensional design models by Partial Least Squares dimension reduction , 2016, Structural and Multidisciplinary Optimization.

[40]  Christodoulos A. Floudas,et al.  Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption , 2017, J. Glob. Optim..

[41]  Christos T. Maravelias,et al.  Surrogate‐based superstructure optimization framework , 2011 .

[42]  Sigurd Skogestad,et al.  Surrogate model generation using self-optimizing variables , 2018, Comput. Chem. Eng..

[43]  Jack P. C. Kleijnen,et al.  Kriging Metamodeling in Simulation: A Review , 2007, Eur. J. Oper. Res..

[44]  J. Cornell.  Experiments with Mixtures: Designs, Models and the Analysis of Mixture Data , 1982 .

[45]  Christos T. Maravelias,et al.  Surrogate-Based Process Synthesis , 2010 .

[46]  Dick den Hertog,et al.  Space-filling Latin hypercube designs for computer experiments , 2008 .

[47]  Alessandro Brambilla,et al.  Optimal operation of a separation plant using artificial neural networks , 1998 .

[48]  Nikolaos V. Sahinidis,et al.  A polyhedral branch-and-cut approach to global optimization , 2005, Math. Program..

[49]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[50]  Ky Khac Vu,et al.  Surrogate-based methods for black-box optimization , 2017, Int. Trans. Oper. Res..

[51]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[52]  Sigurd Skogestad,et al.  A new termination criterion for sampling for surrogate model generation using partial least squares regression , 2019, Comput. Chem. Eng..

[53]  Mohammad Reza Rahimpour,et al.  Progress in catalytic naphtha reforming process: A review , 2013 .

[54]  M. E. Johnson,et al.  Minimax and maximin distance designs , 1990 .

[55]  Dick den Hertog,et al.  Maximin Latin Hypercube Designs in Two Dimensions , 2007, Oper. Res..

[56]  Iftekhar A. Karimi,et al.  Design of computer experiments: A review , 2017, Comput. Chem. Eng..

[57]  V. R. Joseph,et al.  ORTHOGONAL-MAXIMIN LATIN HYPERCUBE DESIGNS , 2008 .

[58]  Peter Goos,et al.  I-Optimal Design of Mixture Experiments , 2016 .

[59]  Sébastien Le Digabel,et al.  Use of quadratic models with mesh-adaptive direct search for constrained black box optimization , 2011, Optim. Methods Softw..

[60]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[61]  G. Valavarasu,et al.  Light Naphtha Isomerization Process: A Review , 2013 .

[62]  Sigurd Skogestad,et al.  Use of Latent Variables to Reduce the Dimension of Surrogate Models , 2017 .

[63]  Andreas Psaltis,et al.  Systematic optimization methodology for heat exchanger network and simultaneous process design , 2016, Comput. Chem. Eng..

[64]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[65]  R. L. Hardy.  Multiquadric equations of topography and other irregular surfaces , 1971 .

[66]  Roberto Guardani,et al.  Neural network based approach for optimization of industrial chemical processes , 2000 .

[67]  Ignacio E. Grossmann,et al.  A review on superstructure optimization approaches in process system engineering , 2020, Comput. Chem. Eng..

[68]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[69]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[70]  Selen Cremaschi,et al.  Process synthesis of biodiesel production plant using artificial neural networks as the surrogate models , 2012, Comput. Chem. Eng..

[71]  Uday T Turaga,et al.  Catalytic Naphtha Reforming: Revisiting its Importance in the Modern Refinery* , 2003 .

[72]  Wendell F. Smith.  Experimental design for formulation , 2005, ASA-SIAM series on statistics and applied probability.

[73]  Martin D. Buhmann,et al.  Radial Basis Functions , 2021, Encyclopedia of Mathematical Geosciences.

[74]  I. Grossmann,et al.  An algorithm for the use of surrogate models in modular flowsheet optimization , 2008 .

[75]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[76]  Mario R. Eden,et al.  Efficient Surrogate Model Development: Impact of Sample Size and Underlying Model Dimensions , 2018 .

[77]  Rachel Ward,et al.  A Unified Framework for Linear Dimensionality Reduction in L1 , 2014, Results in Mathematics.

[78]  Fani Boukouvala,et al.  Machine learning-based surrogate modeling for data-driven optimization: a comparison of subset selection for regression techniques , 2019, Optimization Letters.

[79]  Mark P. Lapinski,et al.  Catalytic Reforming in Petroleum Processing , 2014 .