A hybrid evolutionary algorithm for the symbolic modeling of multiple-time-scale dynamical systems

Natural and artificial dynamical systems in the real world often have dynamics at multiple time scales. Such dynamics can contribute substantially to the complexity of a dynamical system and increase the difficulty with which it can be analyzed. Although evolutionary algorithms have been proposed that are amenable to the automated modeling of dynamical systems, none have explicitly taken into account multiple time scales or leveraged the information about these dynamics that is inherent in experimental observations. We propose a hybrid approach to the design of models for multiple-time-scale dynamical systems that combines an evolutionary algorithm with other metaheuristics and conventional nonlinear regression. With only minimal human-supplied domain knowledge, the algorithm automates the process of analyzing raw experimental observations and creating an interpretable symbolic model of the system under study. We describe the algorithm in detail and demonstrate its applicability to a variety of both physical and simulated systems. In addition, we study the performance and scalability of the algorithm under different types of dynamics, varying levels of experimental noise, and other factors relevant to the practical application of the algorithm.

[1]  Christopher H. Bryant,et al.  Functional genomic hypothesis generation and experimentation by a robot scientist , 2004, Nature.

[2]  Hod Lipson,et al.  Predicting solution rank to improve performance , 2010, GECCO '10.

[3]  Gregory Hornby,et al.  ALPS: the age-layered population structure for reducing the problem of premature convergence , 2006, GECCO.

[4]  James A. Evans,et al.  Machine Science , 2010, Science.

[5]  R. Savit,et al.  Dynamics of genetic programming and chaotic time series prediction , 1996 .

[6]  Bernhard O. Palsson,et al.  On the dynamics of the irreversible Michaelis-Menten reaction mechanism , 1987 .

[7]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[8]  Jim Gray,et al.  2020 Computing: Science in an exponential world , 2006, Nature.

[9]  Hod Lipson,et al.  Coevolution of Fitness Predictors , 2008, IEEE Transactions on Evolutionary Computation.

[10]  J. Fredberg,et al.  Fast and slow dynamics of the cytoskeleton , 2006, Nature materials.

[11]  Hanna Tuomisto,et al.  DISSECTING THE SPATIAL STRUCTURE OF ECOLOGICAL DATA AT MULTIPLE SCALES , 2004 .

[12]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..

[13]  Hitoshi Iba,et al.  Genetic Programming with Local Hill-Climbing , 1994, PPSN.

[14]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[15]  Hod Lipson,et al.  Comparison of tree and graph encodings as function of problem complexity , 2007, GECCO '07.

[16]  Hod Lipson,et al.  Age-fitness pareto optimization , 2010, GECCO '10.

[17]  R. Haddad,et al.  Multiresolution Signal Decomposition: Transforms, Subbands, and Wavelets , 1992 .

[18]  Ken E. Whelan,et al.  The Automation of Science , 2009, Science.

[19]  Aytac Guven,et al.  Linear genetic programming for time-series modelling of daily flow rate , 2009 .

[20]  Kenneth de Jong,et al.  Evolutionary computation: a unified approach , 2007, GECCO.

[21]  F. Verhulst.  Methods and Applications of Singular Perturbations: Boundary Layers and Multiple Timescale Dynamics , 2010 .

[22]  Herbert A. Simon,et al.  Scientific discovery: compulalional explorations of the creative process , 1987 .

[23]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[24]  John R. Koza,et al.  Genetic Programming III: Darwinian Invention & Problem Solving , 1999 .

[25]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[26]  Michael D. Schmidt,et al.  Automated refinement and inference of analytical models for metabolic networks , 2011, Physical biology.

[27]  H. Iba,et al.  Inferring a system of differential equations for a gene regulatory network by using genetic programming , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[28]  N. Draper,et al.  Applied Regression Analysis: Draper/Applied Regression Analysis , 1998 .

[29]  S. Strogatz.  Exploring complex networks , 2001, Nature.

[30]  J. Beisler.  Isolation, characterization, and properties of a labile hydrolysis product of the antitumor nucleoside, 5-azacytidine. , 1978, Journal of medicinal chemistry.

[31]  M. Karplus,et al.  A hierarchy of timescales in protein dynamics is linked to enzyme catalysis , 2007, Nature.

[32]  Samir W. Mahfoud.  Crowding and Preselection Revisited , 1992, PPSN.

[33]  Bruce G Buchanan,et al.  Automating Science , 2009, Science.

[34]  R. Harris-Warrick,et al.  Multiple mechanisms of bursting in a conditional bursting neuron , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[35]  Hans G Othmer,et al.  A multi-time-scale analysis of chemical reaction networks: I. Deterministic systems , 2010, Journal of mathematical biology.

[36]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[37]  J. Fish,et al.  A Dispersive Model for Wave Propagation in Periodic Heterogeneous Media Based on Homogenization With Multiple Spatial and Temporal Scales , 2001 .

[38]  J. Saurina,et al.  Study of the degradation of 5-azacytidine as a model of unstable drugs using a stopped-flow method and further data analysis with multivariate curve resolution. , 2007, Talanta.

[39]  William W. Chen,et al.  Classic and contemporary approaches to modeling biochemical reactions. , 2010, Genes & development.

[40]  Zbigniew Michalewicz,et al.  Time Series Forecasting for Dynamic Environments: The DyFor Genetic Program Model , 2007, IEEE Transactions on Evolutionary Computation.

[41]  T. Sonar,et al.  Asymptotic adaptive methods for multi-scale problems in fluid mechanics , 2001 .

[42]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[43]  Kenneth DeJong.  Evolutionary computation: a unified approach , 2007, GECCO.

[44]  Daniel Caruana,et al.  Control of self-oscillating systems , 2003 .

[45]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[46]  J. Pandolfi,et al.  Coral community dynamics at multiple scales , 2002, Coral Reefs.

[47]  Emiliano Carreño Jara.  Long memory time series forecasting by using genetic programming , 2011 .

[48]  Hod Lipson,et al.  Symbolic regression of multiple-time-scale dynamical systems , 2012, GECCO '12.

[49]  J. M. de St Georges.  All for one and one for all. , 1993, Dental teamwork.

[50]  下平 丕作士,et al.  The Genetic and Evolutionary Computation Conference , 2002 .

[51]  Christopher K. R. T. Jones,et al.  Multiple-Time-Scale Dynamical Systems , 2012 .

[52]  W. Banzhaf,et al.  1 Linear Genetic Programming , 2007 .

[53]  P. Nordin.  Genetic Programming III - Darwinian Invention and Problem Solving , 1999 .

[54]  Hong Hee Yoo,et al.  Stability analysis for the flapwise motion of a cantilever beam with rotary oscillation , 2004 .