Inference of hidden variables in systems of differential equations with genetic programming

The data-driven modeling of dynamical systems is an important scientific activity, and many studies have applied genetic programming (GP) to the task of automatically constructing such models in the form of systems of ordinary differential equations (ODEs). These previous studies assumed that data measurements were available for all variables in the system, whereas in real-world settings, it is typically the case that one or more variables are unmeasured or “hidden.” Here, we investigate the prospect of automatically constructing ODE models of dynamical systems from time series data with GP in the presence of hidden variables. Several examples with both synthetic and physical systems demonstrate the unique challenges of this problem and the circumstances under which it is possible to reverse-engineer both the form and parameters of ODE models with hidden variables.

[1]  E. Lorenz Deterministic nonperiodic flow , 1963 .

[2]  Karl Johan Åström,et al.  BOOK REVIEW SYSTEM IDENTIFICATION , 1994, Econometric Theory.

[3]  K. Glover,et al.  Identifiability of linear and nonlinear dynamical systems , 1976 .

[4]  J. Beisler Isolation, characterization, and properties of a labile hydrolysis product of the antitumor nucleoside, 5-azacytidine. , 1978, Journal of medicinal chemistry.

[5]  James P. Crutchfield,et al.  Geometry from a Time Series , 1980 .

[6]  F. Takens Detecting strange attractors in turbulence , 1981 .

[7]  Pat Langley,et al.  Data-Driven Discovery of Physical Laws , 1981, Cogn. Sci..

[8]  P. Grassberger,et al.  Measuring the Strangeness of Strange Attractors , 1983 .

[9]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[10]  L. D. Kissinger,et al.  Determination of the antileukemia agents cytarabine and azacitidine and their respective degradation products by high-performance liquid chromatography. , 1986, Journal of chromatography.

[11]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[12]  James P. Crutchfield,et al.  Equations of Motion from a Data Series , 1987, Complex Syst..

[13]  Herbert A. Simon,et al.  Scientific discovery: compulalional explorations of the creative process , 1987 .

[14]  J. Cremers,et al.  Construction of Differential Equations from Experimental Data , 1987 .

[15]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[16]  Breeden,et al.  Reconstructing equations of motion from experimental data with unobserved variables. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[17]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[18]  Gouesbet Reconstruction of the vector fields of continuous dynamical systems from numerical scalar time series. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[19]  H. Abarbanel,et al.  Determining embedding dimension for phase-space reconstruction using a geometrical construction. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[20]  Warren E. Stewart,et al.  Parameter estimation from multiresponse data , 1992 .

[21]  Baake,et al.  Fitting ordinary differential equations to chaotic data. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[22]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[23]  J. M. de St Georges All for one and one for all. , 1993, Dental teamwork.

[24]  Howard Oakley,et al.  Two scientific applications of genetic programming: Stack filters and non-linear equation fitting to , 1994 .

[25]  Hitoshi Iba,et al.  Genetic Programming with Local Hill-Climbing , 1994, PPSN.

[26]  H. Rabitz,et al.  IDENTIFIABILITY AND DISTINGUISHABILITY OF GENERAL REACTION SYSTEMS , 1994 .

[27]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[28]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[29]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[30]  Christopher J. Neely,et al.  Is Technical Analysis in the Foreign Exchange Market Profitable? A Genetic Programming Approach , 1996, Journal of Financial and Quantitative Analysis.

[31]  R. Savit,et al.  Dynamics of genetic programming and chaotic time series prediction , 1996 .

[32]  David B. Fogel,et al.  Evolutionary program for the identification of dynamical systems , 1997, Defense, Security, and Sensing.

[33]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[34]  K. Sharman,et al.  Issues in nonlinear model structure identification using genetic programming , 1997 .

[35]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[36]  Saso Dzeroski,et al.  Declarative Bias in Equation Discovery , 1997, ICML.

[37]  David J. Murray-Smith,et al.  Nonlinear model structure identification using genetic programming , 1998 .

[38]  Michael O'Neill,et al.  Grammatical Evolution: Evolving Programs for an Arbitrary Language , 1998, EuroGP.

[39]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[40]  M. Kaboudan Genetic Programming Prediction of Stock Prices , 2000 .

[41]  Takashi Washio,et al.  Enhancing the Plausibility of Law Equation Discovery , 2000, ICML.

[42]  C. Lee Giles,et al.  Learning Chaotic Attractors by Neural Networks , 2000, Neural Computation.

[43]  Pat Langley,et al.  Discovering Communicable Scientific Knowledge from Spatio-Temporal Data , 2001, ICML.

[44]  H. Iba,et al.  Inferring a system of differential equations for a gene regulatory network by using genetic programming , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[45]  L. A. Aguirre,et al.  Structure-selection techniques applied to continous-time nonlinear models , 2001 .

[46]  S. Strogatz Exploring complex networks , 2001, Nature.

[47]  Elizabeth Bradley,et al.  Reasoning about nonlinear system identification , 2001, Artif. Intell..

[48]  Saso Dzeroski,et al.  Inducing Process Models from Continuous Data , 2002, ICML.

[49]  Hitoshi Iba,et al.  Inference of differential equation models by genetic programming , 2002, Inf. Sci..

[50]  Hitoshi Iba,et al.  Evolutionary modeling and inference of gene network , 2002, Inf. Sci..

[51]  S. Ellner,et al.  Rapid evolution drives ecological dynamics in a predator–prey system , 2003, Nature.

[52]  Kwang-Hyun Cho,et al.  Mathematical Modeling of the Influence of RKIP on the ERK Signaling Pathway , 2003, CMSB.

[53]  Markus Brameier,et al.  On linear genetic programming , 2005 .

[54]  R. Adrian,et al.  Transition from laminar to turbulent flow in liquid filled microtubes , 2004 .

[55]  Jürgen Kurths,et al.  Nonlinear Dynamical System Identification from Uncertain and Indirect Measurements , 2004, Int. J. Bifurc. Chaos.

[56]  Lishan Kang,et al.  Evolutionary Modeling of Systems of Ordinary Differential Equations with Genetic Programming , 2000, Genetic Programming and Evolvable Machines.

[57]  Christopher H. Bryant,et al.  Functional genomic hypothesis generation and experimentation by a robot scientist , 2004, Nature.

[58]  Peter J. Fleming,et al.  Evolution of mathematical models of chaotic systems based on multiobjective genetic programming , 2005, Knowledge and Information Systems.

[59]  Takashi Washio,et al.  Scientific Discovery of Dynamic Models Based on Scale-type Constraints , 2005 .

[60]  Stephan M. Winkler,et al.  New methods for the identification of nonlinear model structures based upon genetic programming techniques , 2005 .

[61]  K. Routray,et al.  Kinetic parameter estimation for a multiresponse nonlinear reaction model , 2005 .

[62]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[63]  Jim Gray,et al.  2020 Computing: Science in an exponential world , 2006, Nature.

[64]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[65]  Jens Timmer,et al.  Data-based identifiability analysis of non-linear dynamical models , 2007, Bioinform..

[66]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[67]  J. Saurina,et al.  Study of the degradation of 5-azacytidine as a model of unstable drugs using a stopped-flow method and further data analysis with multivariate curve resolution. , 2007, Talanta.

[68]  Saso Dzeroski,et al.  Inductive process modeling , 2008, Machine Learning.

[69]  Jonathan Becedas,et al.  Algebraic Observer to Estimate Unmeasured State Variables of DC Motors , 2008, Eng. Lett..

[70]  A. Szalay Science in an Exponential World , 2008 .

[71]  Stephen A. Billings,et al.  Model structure selection using an integrated forward orthogonal search algorithm assisted by squared correlation and mutual information , 2008, Int. J. Model. Identif. Control..

[72]  Edward R. Dougherty,et al.  Inference of Noisy Nonlinear Differential Equation Models for Gene Regulatory Networks Using Genetic Programming and Kalman Filtering , 2008, IEEE Transactions on Signal Processing.

[73]  Sheng Chen,et al.  Model selection approaches for non-linear system identification: a review , 2008, Int. J. Syst. Sci..

[74]  Stefano Boccaletti,et al.  The Synchronized Dynamics of Complex Systems , 2008 .

[75]  Hod Lipson,et al.  Coevolution of Fitness Predictors , 2008, IEEE Transactions on Evolutionary Computation.

[76]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[77]  Luis A. Aguirre,et al.  Modeling Nonlinear Dynamics and Chaos: A Review , 2009 .

[78]  Bruce G Buchanan,et al.  Automating Science , 2009, Science.

[79]  Hod Lipson,et al.  Age-fitness pareto optimization , 2010, GECCO '10.

[80]  A Raue,et al.  Identifiability and observability analysis for experimental design in nonlinear dynamical models. , 2010, Chaos.

[81]  James A. Evans,et al.  Machine Science , 2010, Science.

[82]  Abhijit Sen,et al.  A Sniffer Technique for an Efficient Deduction of Model Dynamical Equations Using Genetic Programming , 2011, EuroGP.

[83]  Heder S. Bernardino,et al.  Inferring Systems of Ordinary Differential Equations via Grammar-Based Immune Programming , 2011, ICARIS.

[84]  Christopher J. Neely,et al.  Technical Analysis in the Foreign Exchange Market , 2011 .

[85]  J. Banga,et al.  Structural Identifiability of Systems Biology Models: A Critical Comparison of Methods , 2011, PloS one.

[86]  N. C. King A CRITICAL COMPARISON OF METHODS OF TESTING FOR RESISTANCE TO MOSAIC , .