Extracting macroscopic dynamics: model problems and algorithms

In many applications, the primary objective of numerical simulation of timeevolving systems is the prediction of coarse-grained, or macroscopic, quantities. The purpose of this review is twofold: first, to describe a number of simple model systems where the coarse-grained or macroscopic behaviour of a system can be explicitly determined from the full, or microscopic, description; and second, to overview some of the emerging algorithmic approaches that have been introduced to extract effective, lower-dimensional, macroscopic dynamics. The model problems we describe may be either stochastic or deterministic in both their microscopic and macroscopic behaviour, leading to four possibilities in the transition from microscopic to macroscopic descriptions. Model problems are given which illustrate all four situations, and mathematical tools for their study are introduced. These model problems are useful in the evaluation of algorithms. We use specific instances of the model problems to illustrate these algorithms. As the subject of algorithm development and analysis is, in many cases, in its infancy, the primary purpose here is to attempt to unify some of the emerging ideas so that individuals new to the field have a structured access to the literature. Furthermore, by discussing the algorithms in the context of the model problems, a platform for understanding existing algorithms and developing new ones is built.

[1]  P. Ungar,et al.  Motion under a strong constraining force , 1957 .

[2]  R. Khas'minskii,et al.  Principle of Averaging for Parabolic and Elliptic Differential Equations and for Markov Processes with Small Diffusion , 1963 .

[3]  G. W. Ford,et al.  Statistical Mechanics of Assemblies of Coupled Oscillators , 1965 .

[4]  H. Mori Transport, Collective Motion, and Brownian Motion , 1965 .

[5]  Samuel Karlin,et al.  A First Course on Stochastic Processes , 1968 .

[6]  R. Khas'minskii A Limit Theorem for the Solutions of Differential Equations with Random Right-Hand Sides , 1966 .

[7]  J. Kingman A FIRST COURSE IN STOCHASTIC PROCESSES , 1967 .

[8]  P. Billingsley,et al.  Convergence of Probability Measures , 1970, The Mathematical Gazette.

[9]  Iosif Ilitch Gikhman,et al.  Introduction to the theory of random processes , 1969 .

[10]  Neil Fenichel Persistence and Smoothness of Invariant Manifolds for Flows , 1971 .

[11]  R. Zwanzig Nonlinear generalized Langevin equations , 1973 .

[12]  George Papanicolaou,et al.  A limit theorem with strong mixing in banach space and two applications to stochastic differential equations , 1973 .

[13]  Thomas G. Kurtz,et al.  A limit theorem for perturbed operator semigroups with applications to random evolutions , 1973 .

[14]  Hazime Mori,et al.  A New Expansion of the Master Equation , 1974 .

[15]  V. Arnold Mathematical Methods of Classical Mechanics , 1974 .

[16]  M. Fixman Classical statistical mechanics of constraints: a theorem and application to polymers. , 1974, Proceedings of the National Academy of Sciences of the United States of America.

[17]  D. Gillespie A General Method for Numerically Simulating the Stochastic Time Evolution of Coupled Chemical Reactions , 1976 .

[18]  O. Rössler An equation for continuous chaos , 1976 .

[19]  John C. Wells Invariant manifolds on non-linear operators. , 1976 .

[20]  T. Kurtz Limit theorems and diffusion approximations for density dependent Markov chains , 1976 .

[21]  G. Papanicolaou Some probabilistic problems and methods in singular perturbations , 1976 .

[22]  D. Gillespie Exact Stochastic Simulation of Coupled Chemical Reactions , 1977 .

[23]  M. M. Tropper Ergodic and quasideterministic properties of finite-dimensional stochastic systems , 1977 .

[24]  S. Orszag,et al.  Advanced Mathematical Methods For Scientists And Engineers , 1979 .

[25]  T. Kurtz Strong approximation theorems for density dependent Markov chains , 1978 .

[26]  L. Rogers,et al.  Diffusions, Markov processes, and martingales , 1979 .

[27]  Neil Fenichel Geometric singular perturbation theory for ordinary differential equations , 1979 .

[28]  Robert Zwanzig,et al.  Problems in nonlinear transport theory , 1980 .

[29]  F. Takens Motion under the influence of a strong constraining force , 1980 .

[30]  J. Carr Applications of Centre Manifold Theory , 1981 .

[31]  K. Lindenberg,et al.  Dissipative contributions of internal multiplicative noise: I. mechanical oscillator , 1981 .

[32]  K. Lindenberg,et al.  Dissipative contributions of internal multiplicative noise: II. Spin systems☆ , 1982 .

[33]  C. Gardiner Handbook of Stochastic Methods , 1983 .

[34]  Jack K. Hale,et al.  Infinite dimensional dynamical systems , 1983 .

[35]  A. Neishtadt The separation of motions in systems with rapidly rotating phase , 1984 .

[36]  H. Risken The Fokker-Planck equation : methods of solution and applications , 1985 .

[37]  H. Risken Fokker-Planck Equation , 1984 .

[38]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[39]  J. Kahane Some Random Series of Functions , 1985 .

[40]  F. Verhulst,et al.  Averaging Methods in Nonlinear Dynamical Systems , 1985 .

[41]  N. Kampen,et al.  Elimination of fast variables , 1985 .

[42]  B. Øksendal Stochastic Differential Equations , 1985 .

[43]  Antonio Giorgilli,et al.  Realization of holonomic constraints and freezing of high frequency degrees of freedom in the light of classical perturbation theory. Part II , 1987 .

[44]  G. W. Ford,et al.  On the quantum langevin equation , 1987 .

[45]  Antonio Giorgilli,et al.  Realization of holonomic constraints and freezing of high frequency degrees of freedom in the light of classical perturbation theory. Part I , 1987 .

[46]  C. Meunier,et al.  Multiphase Averaging for Classical Systems , 1988 .

[47]  Ford,et al.  On the quantum langevin equation , 1981, Physical review. A, General physics.

[48]  J. Hale Asymptotic Behavior of Dissipative Systems , 1988 .

[49]  C. Cercignani The Boltzmann equation and its applications , 1988 .

[50]  Lawrence Sirovich,et al.  LOW DIMENSIONAL DESCRIPTION OF COMPLICATED PHENOMENA , 1988 .

[51]  Ernst Hairer,et al.  The numerical solution of differential-algebraic systems by Runge-Kutta methods , 1989 .

[52]  M. Yor DIFFUSIONS, MARKOV PROCESSES AND MARTINGALES: Volume 2: Itô Calculus , 1989 .

[53]  S. Wiggins Introduction to Applied Nonlinear Dynamical Systems and Chaos , 1989 .

[54]  I. Kevrekidis,et al.  Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: analysis and computations , 1990 .

[55]  Christian Beck Brownian motion from deterministic dynamics , 1990 .

[56]  Fox,et al.  Amplification of intrinsic fluctuations by chaotic dynamics in physical systems. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[57]  J. R. E. O’Malley Singular perturbation methods for ordinary differential equations , 1991 .

[58]  K. Nipp,et al.  On the Application of Invariant Manifold Theory, in particular to Numerical Analysis , 1991 .

[59]  Heinz-Otto Kreiss,et al.  Problems with different time scales , 1992, Acta Numerica.

[60]  Ulrich Maas,et al.  Simplifying chemical kinetics: Intrinsic low-dimensional manifolds in composition space , 1992 .

[61]  Yuri Kifer,et al.  Averaging in dynamical systems and large deviations , 1992 .

[62]  Edriss S. Titi,et al.  On the rate of convergence of the nonlinear Galerkin methods , 1993 .

[63]  H. Berendsen,et al.  Essential dynamics of proteins , 1993, Proteins.

[64]  Ernst Hairer,et al.  Solving Ordinary Differential Equations I: Nonstiff Problems , 2009 .

[65]  J. M. Sanz-Serna,et al.  Numerical Hamiltonian Problems , 1994 .

[66]  S. Lam,et al.  The CSP method for simplifying kinetics , 1994 .

[67]  N. Krylov Introduction to the theory of diffusion processes , 1994 .

[68]  Kun Xu,et al.  Numerical Navier-Stokes solutions from gas kinetic theory , 1994 .

[69]  Klaus Schulten,et al.  Coupling of protein motion to electron transfer in a photosynthetic reaction center: investigating the low temperature behavior in the framework of the spin—boson model , 1994 .

[70]  P. Tavan,et al.  Molecular dynamics of conformational substates for a simplified protein model , 1994 .

[71]  T. Hughes Multiscale phenomena: Green's functions, the Dirichlet-to-Neumann formulation, subgrid scale models, bubbles and the origins of stabilized methods , 1995 .

[72]  Y. Kifer Limit theorems in averaging for dynamical systems , 1995, Ergodic Theory and Dynamical Systems.

[73]  Sebastian Reich,et al.  Smoothed dynamics of highly oscillatory Hamiltonian systems , 1995 .

[74]  A. Katok,et al.  Introduction to the Modern Theory of Dynamical Systems: Low-dimensional phenomena , 1995 .

[75]  K. Nipp,et al.  Invariant manifolds and global error estimates of numerical integration schemes applied to stiff systems of singular perturbation type -- Part I: RK-methods , 1995 .

[76]  Peter Deuflhard,et al.  Dynamic Dimension Reduction in ODE Models , 1996 .

[77]  Invariant manifolds and global error estimates of numerical integration schemes applied to stiff systems of singular perturbation type – Part II: Linear multistep methods , 1996 .

[78]  Hans Christian Öttinger,et al.  Stochastic Processes in Polymeric Fluids , 1996 .

[79]  Zvi Artstein,et al.  Singularly perturbed ordinary differential equations with dynamic limits , 1996, Proceedings of the Royal Society of Edinburgh: Section A Mathematics.

[80]  Kinetic limits for a class of interacting particle systems , 1996 .

[81]  K. Schulten Curve Crossing in a Protein: Coupling of the Elementary Quantum Process to Motions of the Protein , 1996 .

[82]  A. R. Humphries,et al.  Dynamical Systems And Numerical Analysis , 1996 .

[83]  Linear Response of Hamiltonian Chaotic Systems as a Function of the Number of Degrees of Freedom. , 1996, Physical review letters.

[84]  P. Holmes,et al.  Turbulence, Coherent Structures, Dynamical Systems and Symmetry , 1996 .

[85]  Linda R. Petzold,et al.  Numerical solution of highly oscillatory ordinary differential equations , 1997, Acta Numerica.

[86]  C. Schütte,et al.  Homogenization of Hamiltonian systems with a strong constraining potential , 1997 .

[87]  C. Pillet,et al.  Ergodic Properties of the Non-Markovian Langevin Equation , 1997 .

[88]  A J Chorin,et al.  Optimal prediction of underresolved dynamics. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[89]  Robert E. Rudd,et al.  COARSE-GRAINED MOLECULAR DYNAMICS AND THE ATOMIC LIMIT OF FINITE ELEMENTS , 1998 .

[90]  R. Skeel,et al.  Nonlinear Resonance Artifacts in Molecular Dynamics Simulations , 1998 .

[91]  E. Titi,et al.  Postprocessing Fourier spectral methods: the case of smooth solutions , 1998 .

[92]  Michael Dellnitz,et al.  An adaptive subdivision technique for the approximation of attractors and invariant measures , 1998 .

[93]  Folkmar Bornemann,et al.  Homogenization in Time of Singularly Perturbed Mechanical Systems , 1998, Lecture notes in mathematics.

[94]  X. Mao,et al.  Stochastic Differential Equations and Applications , 1998 .

[95]  Stochastic modelling of turbulence and anomalous transport in plasmas , 1998 .

[96]  J. Zukas Introduction to the Modern Theory of Dynamical Systems , 1998 .

[97]  Robert D. Skeel,et al.  Long-Time-Step Methods for Oscillatory Differential Equations , 1998, SIAM J. Sci. Comput..

[98]  A. J. Chorin,et al.  Optimal Prediction for Hamiltonian Partial Differential Equations , 1999 .

[99]  Robert N. Miller,et al.  Data assimilation into nonlinear stochastic models , 1999 .

[100]  Prakasa Rao Statistical inference for diffusion type processes , 1999 .

[101]  Vanden Eijnden E,et al.  Models for stochastic climate prediction. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[102]  Michael Dellnitz,et al.  Computation of Essential Molecular Dynamics by Subdivision Techniques , 1996, Computational Molecular Dynamics.

[103]  Andrew M. Stuart,et al.  Analysis and Experiments for a Computational Model of a Heat Bath , 1999 .

[104]  Benedict Leimkuhler,et al.  Computational Molecular Dynamics: Challenges, Methods, Ideas , 1999, Computational Molecular Dynamics.

[105]  O H Hald Optimal prediction and the Klein-Gordon equation. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[106]  Uri M. Ascher,et al.  The Midpoint Scheme and Variants for Hamiltonian Systems: Advantages and Pitfalls , 1999, SIAM J. Sci. Comput..

[107]  P. Deuflhard,et al.  A Direct Approach to Conformational Dynamics Based on Hybrid Monte Carlo , 1999 .

[108]  R. Freund Reduced-Order Modeling Techniques Based on Krylov Subspaces and Their Use in Circuit Simulation , 1999 .

[109]  O. Junge,et al.  On the Approximation of Complicated Dynamical Behavior , 1999 .

[110]  D. Sorensen,et al.  A Survey of Model Reduction Methods for Large-Scale Systems , 2000 .

[111]  Sebastian Reich,et al.  Smoothed Langevin dynamics of highly oscillatory systems , 2000 .

[112]  P. Deuflhard,et al.  Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains , 2000 .

[113]  I. Kevrekidis,et al.  "Coarse" stability and bifurcation analysis using time-steppers: a reaction-diffusion example. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[114]  A J Majda,et al.  Remarkable statistical behavior for truncated Burgers-Hopf dynamics. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[115]  Thomas Y. Hou,et al.  Convergence of a Nonconforming Multiscale Finite Element Method , 2000, SIAM J. Numer. Anal..

[116]  A P Kast Optimal prediction of stiff oscillatory mechanics. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[117]  A J Chorin,et al.  Optimal prediction and the Mori-Zwanzig representation of irreversible processes. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[118]  Alexandre J. Chorin,et al.  Stochastic Optimal Prediction with Application to Averaged Euler Equations , 2000, COLING 2000.

[119]  Ioannis G. Kevrekidis,et al.  “Coarse” stability and bifurcation analysis using stochastic simulators: Kinetic Monte Carlo examples , 2001, nlin/0111038.

[120]  Y. Kifer Averaging and climate models , 2001 .

[121]  Andrew J. Majda,et al.  A mathematical framework for stochastic climate models , 2001 .

[122]  Marshall Slemrod,et al.  On Singularly Perturbed Retarded Functional Differential Equations , 2001 .

[123]  H. Kantz,et al.  Stochastic modelling: replacing fast degrees of freedom by noise , 2001 .

[124]  T. Palmer A nonlinear dynamical perspective on model error: A proposal for non‐local stochastic‐dynamic parametrization in weather and climate prediction models , 2001 .

[125]  W. E,et al.  Matching conditions in atomistic-continuum modeling of materials. , 2001, Physical review letters.

[126]  Endre Süli,et al.  Stiff Oscillatory Systems, Delta Jumps and White Noise , 2001, Found. Comput. Math..

[127]  E Weinan,et al.  A dynamic atomistic-continuum method for the simulation of crystalline materials , 2001 .

[128]  Yuri Kifer,et al.  STOCHASTIC VERSIONS OF ANOSOV'S AND NEISTADT'S THEOREMS ON AVERAGING , 2001 .

[129]  Andrew J. Majda,et al.  A priori tests of a stochastic mode reduction strategy , 2002 .

[130]  G. Golub,et al.  Computation of large-scale quadratic forms and transfer functions using the theory of moments, quadrature and Padé approximation , 2002 .

[131]  Zvi Artstein,et al.  ON SINGULARLY PERTURBED ORDINARY DIFFERENTIAL EQUATIONS WITH MEASURE-VALUED LIMITS , 2002 .

[132]  E. Hairer,et al.  Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems , 2010 .

[133]  C. W. Gear,et al.  'Coarse' integration/bifurcation analysis via microscopic simulators: Micro-Galerkin methods , 2002 .

[134]  Klaus Schulten,et al.  Excitons in a photosynthetic light-harvesting system: a combined molecular dynamics, quantum chemistry, and polaron model study. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[135]  Theory of phase separation kinetics in polymer – liquid crystal systems , 2002 .

[136]  A. Majdaa,et al.  A priori tests of a stochastic mode reduction strategy , 2002 .

[137]  Alexandre J. Chorin,et al.  Optimal prediction with memory , 2002 .

[138]  Paul F. Tupper,et al.  LONG-TERM BEHAVIOUR OF LARGE MECHANICAL SYSTEMS WITH RANDOM INITIAL DATA , 2002 .

[139]  T. Chan,et al.  Multiscale and multiresolution methods : theory and applications , 2002 .

[140]  Dimitri D. Vvedensky,et al.  Fluctuations in the lattice gas for Burgers' equation , 2002 .

[141]  Constantinos Theodoropoulos,et al.  Equation-Free Multiscale Computation: enabling microscopic simulators to perform system-level tasks , 2002 .

[142]  Ole H. Hald,et al.  Asymptotic and Numerical Analyses for Mechanical Models of Heat Baths , 2002 .

[143]  Ioannis G. Kevrekidis,et al.  Coarse bifurcation analysis of kinetic Monte Carlo simulations: A lattice-gas model with lateral interactions , 2002 .

[144]  Weiqing Ren,et al.  Higher Order String Method for Finding Minimum Energy Paths , 2003 .

[145]  Wolfram Just,et al.  Elimination of Fast Chaotic Degrees of Freedom: On the Accuracy of the Born Approximation , 2003 .

[146]  C. W. Gear,et al.  Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis , 2003 .

[147]  Wilhelm Huisinga,et al.  Extracting macroscopic stochastic dynamics: Model problems , 2003 .

[148]  I. Kevrekidis,et al.  Coarse molecular dynamics of a peptide fragment: Free energy, kinetics, and long-time dynamics computations , 2002, physics/0212108.

[149]  Eric Vanden-Eijnden,et al.  NUMERICAL TECHNIQUES FOR MULTI-SCALE DYNAMICAL SYSTEMS WITH STOCHASTIC EFFECTS ⁄ , 2003 .

[150]  Averaging and renormalization for the KdV-burgers equation , 2003 .

[151]  Andrew J. Majda,et al.  Closure Approximations for Passive Scalar Turbulence: A Comparative Study on an Exactly Solvable Model with Complex Features , 2003 .

[152]  D. Vvedensky Edwards-Wilkinson equation from lattice transition rules. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[153]  E. Weinan Analysis of the heterogeneous multiscale method for ordinary differential equations , 2003 .

[154]  Alexandre J Chorin,et al.  Averaging and renormalization for the Korteveg–deVries–Burgers equation , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[155]  Andrew J. Majda,et al.  Systematic Strategies for Stochastic Mode Reduction in Climate , 2003 .

[156]  E Weinan,et al.  The Heterognous Multiscale Methods , 2003 .

[157]  Ioannis G. Kevrekidis,et al.  Projective Methods for Stiff Differential Equations: Problems with Gaps in Their Eigenvalue Spectrum , 2002, SIAM J. Sci. Comput..

[158]  Andrew M. Stuart,et al.  Fitting SDE models to nonlinear Kac–Zwanzig heat bath models , 2004 .

[159]  Paul Tavan,et al.  Topological feature maps with self-organized lateral connections: a population-coded, one-layer model of associative memory , 1994, Biological Cybernetics.

[160]  Pingwen Zhang,et al.  Stochastic models of polymeric fluids at small Deborah number , 2004 .

[161]  R. Kupferman Fractional Kinetics in Kac–Zwanzig Heat Bath Models , 2004 .

[162]  R. Kupferman,et al.  White noise limits for discrete dynamical systems driven by fast deterministic dynamics , 2004 .

[163]  R. Kupfermana,et al.  Fitting SDE models to nonlinear Kac – Zwanzig heat bath models , 2004 .

[164]  Benjamin Jourdain,et al.  Existence of solution for a micro–macro model of polymeric fluid: the FENE model , 2004 .

[165]  Dror Givon,et al.  Existence proof for orthogonal dynamics and the Mori-Zwanzig formalism , 2005 .

[166]  Paul F. Tupper,et al.  A test problem for molecular dynamics integrators , 2005 .

[167]  E. Vanden-Eijnden,et al.  Analysis of multiscale methods for stochastic differential equations , 2005 .

[168]  E. Hairer,et al.  Geometric Numerical Integration , 2022, Oberwolfach Reports.

[169]  G. W. FonDt Statistical Mechanics of Assemblies of Coupled Oscillators * , 2022 .