Artificial Neural Networks for dynamic optimization of stochastic multiscale systems subject to uncertainty

Abstract The purpose of this study was to employ Artificial Neural Networks (ANNs) to develop data-driven models that would enable optimal control of a stochastic multiscale system subject to parametric uncertainty. The system used for the case study was a simulation of thin film formation by chemical vapour deposition, where a solid-on-solid kinetic Monte Carlo model was coupled with continuum transport equations. The ANNs were trained to estimate the dynamic responses of statistical moments of the system’s observables and subsequently employed in a dynamic optimization scheme to identify the optimal profiles of the manipulated variables that would attain the desired thin film properties at the end of the batch. The resulting profiles were validated using the stochastic multiscale system and a close agreement with ANN-based predictions was observed. Due to their computational efficiency, accuracy, and the ability to reject disturbances, the ANNs appear to be an attractive approach for the optimization and control of computationally demanding multiscale process systems.

[1]  Shabnam Rasoulian,et al.  A robust nonlinear model predictive controller for a multiscale thin film deposition process , 2015 .

[2]  M. Stamatakis,et al.  A review of multiscale modeling of metal-catalyzed reactions: Mechanism development for complexity and emergent behavior , 2011 .

[3]  Eric Croiset,et al.  Carbon nanotube growth: First-principles-based kinetic Monte Carlo model , 2015 .

[4]  P. R. Westmoreland,et al.  Hierarchical Reduced Models for Catalytic Combustion: H2/Air Mixtures Near Platinum Surfaces , 1997 .

[5]  Huang Kai,et al.  Catalyst design for methane oxidative coupling by using artificial neural network and hybrid genetic algorithm , 2003 .

[6]  P. Christofides,et al.  Multiscale modeling and operation of PECVD of thin film solar cells , 2015 .

[7]  J. Kwon,et al.  Koopman Lyapunov‐based model predictive control of nonlinear chemical process systems , 2019, AIChE Journal.

[8]  Prashant Mhaskar,et al.  Subspace Identification Based Modeling and Control of Batch Particulate Processes , 2017, Modeling and Control of Batch Processes.

[9]  Berend Smit,et al.  Understanding molecular simulation: from algorithms to applications , 1996 .

[10]  Panagiotis D. Christofides,et al.  Multiscale computational fluid dynamics modeling of thermal atomic layer deposition with application to chamber design , 2019, Chemical Engineering Research and Design.

[11]  J. Kwon,et al.  Multiscale modeling and control of Kappa number and porosity in a batch‐type pulp digester , 2019, AIChE Journal.

[12]  Dionisios G. Vlachos,et al.  Low-Dimensional Approximations of Multiscale Epitaxial Growth Models for Microstructure Control of Materials , 2000 .

[13]  Kan Wu,et al.  Optimization of simultaneously propagating multiple fractures in hydraulic fracturing to achieve uniform growth using data-based model reduction , 2018, Chemical Engineering Research and Design.

[14]  Curtis B. Storlie,et al.  Upscaling Uncertainty with Dynamic Discrepancy for a Multi-Scale Carbon Capture System , 2014 .

[15]  P. Christofides,et al.  Multiscale modeling and run-to-run control of PECVD of thin film solar cells , 2017 .

[16]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[17]  Geoffrey E. Hinton,et al.  Temporal-Kernel Recurrent Neural Networks , 2010, Neural Networks.

[18]  Shabnam Rasoulian,et al.  Robust multivariable estimation and control in an epitaxial thin film growth process under uncertainty , 2015 .

[19]  A. Boudouvis,et al.  Multiscale modeling in chemical vapor deposition processes: Coupling reactor scale with feature scale computations , 2010 .

[20]  Venkat Venkatasubramanian,et al.  Design of Fuel Additives Using Neural Networks and Evolutionary Algorithms , 2001 .

[21]  T. Trucano,et al.  Verification, Validation, and Predictive Capability in Computational Engineering and Physics , 2004 .

[22]  Joseph Sang-Il Kwon,et al.  Data-driven identification of interpretable reduced-order models using sparse regression , 2018, Comput. Chem. Eng..

[23]  Panagiotis D. Christofides,et al.  Multiscale, Multidomain Modeling and Parallel Computation: Application to Crystal Shape Evolution in Crystallization , 2015 .

[24]  Panagiotis D. Christofides,et al.  Dynamics and control of aggregate thin film surface morphology for improved light trapping: Implementation on a large-lattice kinetic Monte Carlo model , 2011 .

[25]  Joseph Sang-Il Kwon,et al.  Identification of cell‐to‐cell heterogeneity through systems engineering approaches , 2020 .

[26]  H. H. Thodberg,et al.  Optimal minimal neural interpretation of spectra , 1992 .

[27]  D. Vlachos.  Multiscale integration hybrid algorithms for homogeneous–heterogeneous reactors , 1997 .

[28]  Luo,et al.  Surface roughness and conductivity of thin Ag films. , 1994, Physical review. B, Condensed matter.

[29]  Dionisios G. Vlachos,et al.  Multiscale model for epitaxial growth of films: Growth mode transition , 2001 .

[30]  Sergey Oladyshkin,et al.  Data-driven uncertainty quantification using the arbitrary polynomial chaos expansion , 2012, Reliab. Eng. Syst. Saf..

[31]  D. Vlachos,et al.  Catalytic ignition and extinction of hydrogen: comparison of simulations and experiments , 1996 .

[32]  Panagiotis D. Christofides,et al.  A method for handling batch-to-batch parametric drift using moving horizon estimation: Application to run-to-run MPC of batch crystallization , 2015 .

[33]  Charles C. Solvason.  Integrated Multiscale Chemical Product Design using Property Clustering and Decomposition Techniques in a Reverse Problem Formulation , 2011 .

[34]  J. Kwon,et al.  Multiscale modeling and multiobjective control of wood fiber morphology in batch pulp digester , 2020 .

[35]  Panagiotis D. Christofides,et al.  Crystal shape and size control using a plug flow crystallization configuration , 2014 .

[36]  D. Vlachos.  A Review of Multiscale Analysis: Examples from Systems Biology, Materials Engineering, and Other Fluid–Surface Interacting Systems , 2005 .

[37]  Panagiotis D. Christofides,et al.  Multiscale three-dimensional CFD modeling for PECVD of amorphous silicon thin films , 2018, Comput. Chem. Eng..

[38]  Grigoriy Kimaev,et al.  A comparison of efficient uncertainty quantification techniques for stochastic multiscale systems , 2017 .

[39]  Luis A. Ricardez-Sandoval,et al.  Nonlinear model predictive control of a multiscale thin film deposition process using artificial neural networks , 2019, Chemical Engineering Science.

[40]  Venkat Venkatasubramanian,et al.  DIAGNOSING NOISY PROCESS DATA USING NEURAL NETWORKS , 1992 .

[41]  Antonios Armaou,et al.  Control and optimization of multiscale process systems , 2006, Comput. Chem. Eng..

[42]  Clarence W. Rowley,et al.  A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition , 2014, Journal of Nonlinear Science.

[43]  M. Eldred.  Recent Advances in Non-Intrusive Polynomial Chaos and Stochastic Collocation Methods for Uncertainty Analysis and Design , 2009 .

[44]  Daniel Svozil,et al.  Introduction to multi-layer feed-forward neural networks , 1997 .

[45]  D. V. Schroeder,et al.  An Introduction to Thermal Physics , 2000 .

[46]  Maciej Ławryńczuk.  Nonlinear predictive control of dynamic systems represented by Wiener–Hammerstein models , 2016 .

[47]  Muthanna H. Al-Dahhan,et al.  Development of an artificial neural network correlation for prediction of overall gas holdup in bubble column reactors , 2003 .

[48]  Thomas A. Adams,et al.  Subspace model identification and model predictive control based cost analysis of a semicontinuous distillation process , 2017, Comput. Chem. Eng..

[49]  Venkat Venkatasubramanian,et al.  A neural network methodology for process fault diagnosis , 1989 .

[50]  Joseph Sang-Il Kwon,et al.  Kinetic Monte Carlo modeling of multivalent binding of CTB proteins with GM1 receptors , 2018, Comput. Chem. Eng..

[51]  Jeroen P. van der Sluijs,et al.  A framework for dealing with uncertainty due to model structure error , 2004 .

[52]  Biao Huang,et al.  Output feedback model predictive control for nonlinear systems represented by Hammerstein-Wiener model , 2007 .

[53]  Mario R. Eden,et al.  Multi-Scale Chemical Product Design using the Reverse Problem Formulation , 2010 .

[54]  Frank Allgöwer,et al.  Analysis of heterogeneous cell populations: A density-based modeling and identification framework , 2011 .

[55]  Fabian J. Theis,et al.  Data-driven modelling of biological multi-scale processes , 2015, 1506.06392.

[56]  Shabnam Rasoulian,et al.  Stochastic nonlinear model predictive control applied to a thin film deposition process under uncertainty , 2016 .

[57]  Jack P. C. Kleijnen,et al.  Regression and Kriging metamodels with their experimental designs in simulation: A review , 2017, Eur. J. Oper. Res..

[58]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[59]  Frank Allgöwer,et al.  Identification of models of heterogeneous cell populations from population snapshot data , 2011, BMC Bioinformatics.

[60]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[61]  Jiali Li,et al.  Recurrent Neural Network-based Model Predictive Control for Continuous Pharmaceutical Manufacturing , 2018, Mathematics.

[62]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[63]  Dionisios G. Vlachos,et al.  Homogeneous-heterogeneous oxidation reactions over platinum and inert surfaces , 1996 .

[64]  Shabnam Rasoulian,et al.  Distributional uncertainty analysis and robust optimization in spatially heterogeneous multiscale process systems , 2016 .

[65]  Ahmet Demir,et al.  Neural network prediction model for the methane fraction in biogas from field-scale landfill bioreactors , 2007, Environ. Model. Softw..

[66]  Victor Picheny,et al.  Comparison of Kriging-based algorithms for simulation optimization with heterogeneous noise , 2017, Eur. J. Oper. Res..

[67]  Prashant Mhaskar,et al.  Subspace-based model identification of a hydrogen plant startup dynamics , 2017, Comput. Chem. Eng..

[68]  Noor Quddus,et al.  Developing Quantitative Structure–Property Relationship Models To Predict the Upper Flammability Limit Using Machine Learning , 2019, Industrial & Engineering Chemistry Research.

[69]  R. Adomaitis.  A reduced-basis discretization method for chemical vapor deposition reactor simulation , 2003 .

[70]  Grigoriy Kimaev,et al.  Multilevel Monte Carlo for noise estimation in stochastic multiscale systems , 2018, Chemical Engineering Research and Design.

[71]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[72]  V. Venkatasubramanian.  The promise of artificial intelligence in chemical engineering: Is it here, finally? , 2018, AIChE Journal.

[73]  Luis A. Ricardez-Sandoval,et al.  Optimization and control of a thin film growth process: A hybrid first principles/artificial neural network based multiscale modelling approach , 2018, Comput. Chem. Eng..

[74]  R. Adomaitis.  Multiscale modeling and optimization of an atomic layer deposition process for nanomanufacturing applications , 2010 .

[75]  Raymond A. Adomaitis,et al.  Development of a multiscale model for an atomic layer deposition process , 2010 .

[76]  Panagiotis D. Christofides,et al.  Modeling and control of crystal shape in continuous protein crystallization , 2014 .

[77]  Panagiotis D. Christofides,et al.  Modeling and Control of Protein Crystal Shape and Size in Batch Crystallization , 2013 .

[78]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1994 .

[79]  Shabnam Rasoulian,et al.  Uncertainty analysis and robust optimization of multiscale process systems with application to epitaxial thin film growth , 2014 .