Simultaneous parameter identification and discrimination of the nonparametric structure of hybrid semi-parametric models

Abstract In this work, a hybrid semi-parametric modelling framework implemented using mixed integer linear programming (MILP) is used to extract (coupled) nonlinear ordinary differential equations (ODEs) from process data. Applied to fed-batch (bio) chemical reaction systems, unknown (or partially known) system connectivity and/or reaction kinetics are represented using a multivariate rational function (MRF) superstructure. The MRF’s are embedded within an ODE framework which is used to incorporate known system model characteristics. Using derivative estimation, the ODEs are decoupled and a MILP algorithm is then used to identify appropriate constitutive model terms using sparse regression. Superstructure sparsity is promoted using a L 0 – pseudo norm penalty, i.e. the cardinality of the model parameter vector, enabling the simultaneous yet decoupled identification of the parameters and model structure discrimination. Using simulated data, two case studies demonstrate a principled approach to hybrid model development, distilling unknown elements of (bio) chemical model structures from process data.

[1]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory, Second Edition , 2000, Statistics for Engineering and Information Science.

[2]  Arthur Richards,et al.  Fast model predictive control with soft constraints , 2013, 2013 European Control Conference (ECC).

[3]  J. Maciejowski,et al.  Soft constraints and exact penalty functions in model predictive control , 2000 .

[4]  Lyle H. Ungar,et al.  A hybrid neural network‐first principles approach to process modeling , 1992 .

[5]  Rui Oliveira.  Combining first principles modelling and artificial neural networks: a general framework , 2004, Comput. Chem. Eng..

[6]  Plamen Angelov,et al.  Hybrid modelling of biotechnological processes using neural networks , 1999 .

[7]  Wolfgang Marquardt,et al.  Incremental identification of hybrid process models , 2008, Comput. Chem. Eng..

[8]  Philippe Bogaerts,et al.  Biological reaction modeling using radial basis function networks , 2004, Comput. Chem. Eng..

[9]  W. Marquardt,et al.  Incremental and simultaneous identification of reaction kinetics: methods and comparison , 2004 .

[10]  J. Shao.  Linear Model Selection by Cross-validation , 1993 .

[11]  Sebastião Feyo de Azevedo,et al.  Hybrid semi-parametric modeling in process systems engineering: Past, present and future , 2014, Comput. Chem. Eng..

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  D. Kamenski,et al.  Parameter estimation in differential equations by application of rational functions , 1993 .

[14]  Joaquin F. Perez-Benito,et al.  The kinetic rate law for autocatalytic reactions , 1987 .

[15]  E. García-Calvo,et al.  Comparison of analysis methods for determination of the kinetic parameters in batch cultures , 2000 .

[16]  W. Cleland,et al.  The statistical analysis of enzyme kinetic data. , 1967, Advances in enzymology and related areas of molecular biology.

[17]  S. Nash,et al.  Linear and Nonlinear Optimization , 2008 .

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Lorenz T. Biegler,et al.  Constraint handing and stability properties of model‐predictive control , 1994 .

[20]  W. Fred Ramirez,et al.  Optimization of Fed‐Batch Bioreactors Using Neural Network Parameter Function Models , 1996 .

[21]  Fang Yao,et al.  Structured functional additive regression in reproducing kernel Hilbert spaces , 2014, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[22]  Maria do Carmo Nicoletti,et al.  Computational Intelligence Techniques for Bioprocess Modelling, Supervision and Control , 2009 .

[23]  Dominic P. Searson,et al.  Inference of chemical reaction networks , 2008 .

[24]  Wolfgang Marquardt,et al.  Experimental design for the identification of hybrid reaction models from transient data , 2008 .

[25]  Mark A. Kramer,et al.  Modeling chemical processes using prior knowledge and neural networks , 1994 .

[26]  Dominique Bonvin,et al.  Target factor analysis for the identification of stoichiometric models , 1990 .

[27]  R. Tibshirani,et al.  On the “degrees of freedom” of the lasso , 2007, 0712.0881.

[28]  Rik Pintelon,et al.  An Introduction to Identification , 2001 .

[29]  Bernold Fiedler,et al.  Local identification of scalar hybrid models with tree structure , 2008 .

[30]  L. Hosten,et al.  A comparative study of short cut procedures for parameter estimation in differential equations , 1979 .

[31]  Jonas S. Almeida,et al.  Decoupling dynamical systems for pathway identification from metabolic profiles , 2004, Bioinform..

[32]  H. Akaike.  A new look at the statistical model identification , 1974 .

[33]  T. Hesterberg,et al.  Least angle and ℓ1 penalized regression: A review , 2008, 0802.0964.

[34]  Mark J. Willis,et al.  Inference of chemical reaction networks using mixed integer linear programming , 2016, Comput. Chem. Eng..

[35]  Rimvydas Simutis,et al.  Hybrid modelling of yeast production processes – combination of a priori knowledge on different levels of sophistication , 1994 .

[36]  A. Richards.  Fast model predictive control with soft constraints , 2013, ECC.

[37]  Dominique Bonvin,et al.  Incremental Identification of Kinetic Models for Homogeneous Reaction Systems , 2006 .

[38]  Wolfgang Marquardt,et al.  Model-Based Experimental Analysis of Kinetic Phenomena in Multi-Phase Reactive Systems , 2005 .

[39]  Gheorghe Maria,et al.  A Review of Algorithms and Trends in Kinetic Model Identification for Chemical and Biochemical Systems , 2004 .

[40]  Athel Cornish-Bowden,et al.  Analysis of enzyme kinetic data , 1995 .

[41]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[42]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.