An Artificial Neural Network approximation based decomposition approach for parameter estimation of system of ordinary differential equations

In this work a new approach for parameter estimation which is based upon decomposing the problem into two subproblems is proposed, the first subproblem generates an Artificial Neural Network (ANN) model from the given data and then the second subproblem uses the ANN model to obtain an estimate of the parameters. The analytical derivates from the ANN model obtained from the first subproblem are used for obtaining the differential terms in the formulation of the second subproblem. This greatly simplifies the parameter estimation problem. The key advantage of the proposed approach is that solution of a large optimization problem requiring high computational resources is avoided and instead two smaller problems are solved. This approach is particularly useful for large and noisy data sets and nonlinear models where ANN models are known to perform quite well and therefore plays an important role in the solution of the overall parameter estimation problem.

[1]  G. Maria.  An adaptive strategy for solving kinetic model concomitant estimation — Reduction problems , 1989 .

[2]  Efstratios N. Pistikopoulos,et al.  On the global and efficient solution of stochastic batch plant design problems , 1997 .

[3]  Robert E. Kalaba,et al.  Quasilinearization and the estimation of chemical rate constants from raw kinetic data [by] R. Bellman [and others] , 1967 .

[4]  D. Wolf,et al.  Estimating rate constants of heterogeneous catalytic reactions without supposition of rate determining surface steps — an application of a genetic algorithm , 1997 .

[5]  Mark S. Leeson,et al.  Intelligent systems : techniques and applications , 2008 .

[6]  R. Luus,et al.  Simplification of quasilinearization method for parameter estimation , 1983 .

[7]  Ignacio E. Grossmann,et al.  Global optimization of nonlinear generalized disjunctive programming with bilinear equality constraints: applications to process networks , 2003, Comput. Chem. Eng..

[8]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[9]  Mark J. O'Malley,et al.  A Lagrangian augmented Hopfield network for mixed integer non-linear programming problems , 2002, Neurocomputing.

[10]  C. Floudas,et al.  Global Optimization for the Parameter Estimation of Differential-Algebraic Systems , 2000 .

[11]  G. Froment,et al.  A hybrid genetic algorithm for the estimation of parameters in detailed kinetic models , 1998 .

[12]  Miroslav Fikar,et al.  Global optimization for parameter estimation of differential-algebraic systems , 2009 .

[13]  I. D. L. Bogle,et al.  Global optimization of modular process flowsheets , 2000 .

[14]  David Kendrick,et al.  GAMS, a user's guide , 1988, SGNM.

[15]  Claire S. Adjiman,et al.  A Rigorous Global Optimization Algorithm for Problems with Ordinary Differential Equations , 2002, J. Glob. Optim..

[16]  Maria Rodriguez-Fernandez,et al.  A hybrid approach for efficient and robust parameter estimation in biochemical pathways. , 2006, Bio Systems.

[17]  Xing-Bao Gao,et al.  A novel neural network for nonlinear convex programming , 2004, IEEE Trans. Neural Networks.

[18]  Venkat Venkatasubramanian,et al.  A hybrid genetic algorithm for efficient parameter estimation of large kinetic models , 2004, Comput. Chem. Eng..

[19]  J. Seinfeld,et al.  A new algorithm for the estimation of parameters in ordinary differential equations , 1972 .

[20]  Leo Liberti,et al.  An Exact Reformulation Algorithm for Large Nonconvex NLPs Involving Bilinear Terms , 2006, J. Glob. Optim..

[21]  Efstratios N. Pistikopoulos,et al.  Parametric controllers in simultaneous process and control design optimization , 2003 .

[22]  D. R. Baughman,et al.  Neural Networks in Bioprocessing and Chemical Engineering , 1992 .

[23]  M. Stadtherr,et al.  Deterministic global optimization of nonlinear dynamic systems , 2007 .

[24]  T. Edgar,et al.  A sequential error-in-variables method for nonlinear dynamic systems , 1991 .

[25]  V. Dua.  A mixed-integer programming approach for optimal configuration of artificial neural networks , 2010 .

[26]  Sohrab Effati,et al.  A new nonlinear neural network for solving convex nonlinear programming problems , 2005, Appl. Math. Comput..

[27]  John Villadsen,et al.  A family of collocation based methods for parameter estimation in differential equations , 1982 .

[28]  James W. Taylor,et al.  Global dynamic optimization for parameter estimation in chemical kinetics. , 2006, The journal of physical chemistry. A.

[29]  Michael P. Walsh,et al.  Augmented Hopfield network for mixed-integer programming , 1999, IEEE Trans. Neural Networks.

[30]  L. Biegler,et al.  Simultaneous solution and optimization strategies for parameter estimation of differential-algebraic equation systems , 1991 .

[31]  Sandro Macchietto,et al.  The optimal design of dynamic experiments , 1989 .

[32]  Venkat Venkatasubramanian,et al.  A neural network methodology for process fault diagnosis , 1989 .

[33]  N. Kalogerakis,et al.  Applied parameter estimation for chemical engineers , 2000 .

[34]  Mohd Azlan Hussain,et al.  Neural Network Techniques and Applications in Chemical Process Control Systems , 2002 .

[35]  J. Villadsen,et al.  Solution of differential equation models by polynomial approximation , 1978 .

[36]  J. Varah.  A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations , 1982 .

[37]  Tiesong Hu,et al.  A neural network approach for solving nonlinear bilevel programming problem , 2008, Comput. Math. Appl..

[38]  L. Hellinckx,et al.  A new method for the estimation of parameters in differential equations , 1974 .

[39]  Yee Leung,et al.  A high-performance feedback neural network for solving convex nonlinear programming problems , 2003, IEEE Trans. Neural Networks.

[40]  Kwong-Sak Leung,et al.  A Neural Network for Solving Nonlinear Programming Problems , 2002, Neural Computing & Applications.

[41]  A. Neumaier,et al.  A global optimization method, αBB, for general twice-differentiable constrained NLPs — I. Theoretical advances , 1998 .

[42]  Rein Luus,et al.  Increasing the size of region of convergence for parameter estimation through the use of shorter data-length , 1980 .

[43]  L. Lasdon,et al.  Efficient data reconciliation and estimation for dynamic processes using nonlinear programming techniques , 1992 .

[44]  R. Sargent,et al.  Solution of a Class of Multistage Dynamic Optimization Problems. 2. Problems with Path Constraints , 1994 .

[45]  D. Himmelblau.  Accounts of Experiences in the Application of Artificial Neural Networks in Chemical Engineering , 2008 .

[46]  Efstratios N. Pistikopoulos,et al.  Global Optimization Issues in Multiparametric Continuous and Mixed-Integer Optimization Problems , 2004, J. Glob. Optim..

[47]  Eric S. Fraga,et al.  Evaluation of hybrid optimization methods for the optimal design of heat integrated distillation sequences , 2003 .

[48]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[49]  Claire S. Adjiman,et al.  Global optimization of dynamic systems , 2004, Comput. Chem. Eng..

[50]  James O. Ramsay,et al.  Selecting optimal weighting factors in iPDA for parameter estimation in continuous-time dynamic models , 2008, Comput. Chem. Eng..

[51]  Wolfgang Marquardt,et al.  Incremental Global Parameter Estimation in Dynamical Systems , 2009 .

[52]  C. Adjiman,et al.  A global optimization method, αBB, for general twice-differentiable constrained NLPs—II. Implementation and computational results , 1998 .

[53]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[54]  Xavier Joulia,et al.  A general strategy for parameter estimation in differential—algebraic systems , 1993 .

[55]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[56]  Sohrab Effati,et al.  A new nonlinear neural network for solving quadratic programming problems , 2005, Appl. Math. Comput..

[57]  Nikolaos V. Sahinidis,et al.  A polyhedral branch-and-cut approach to global optimization , 2005, Math. Program..

[58]  M. Stadtherr,et al.  Deterministic Global Optimization for Parameter Estimation of Dynamic Systems , 2006 .