A review of recent developments in solving ODEs

Mathematical models when simulating the behavior of physical, chemical, and biological systems often include one or more ordinary differential equations (ODEs). To study the system behavior predicted by a model, these equations are usually solved numerically. Although many of the current methods for solving ODEs were developed around the turn of the century, the past 15 years or so has been a period of intensive research. The emphasis of this survey is on the methods and techniques used in software for solving ODEs. ODEs can be classified as stiff or nonstiff, and may be stiff for some parts of an interval and nonstiff for others. We discuss stiff equations, why they are difficult to solve, and methods and software for solving both nonstiff and stiff equations. We conclude this review by looking at techniques for dealing with special problems that may arise in some ODEs, for example, discontinuities. Although important theoretical developments have also taken place, we report only those developments which have directly affected the software and provide a review of this research. We present the basic concepts involved but assume that the reader has some background in numerical computing, such as a first course in numerical methods.

[1]  C. W. Gear,et al.  Automatic Multirate Methods for Ordinary Differential Equation , 1980, IFIP Congress.

[2]  Lawrence F. Shampine,et al.  Implementation of Rosenbrock Methods , 1982, TOMS.

[3]  Kevin Burrage,et al.  An implementation of singly-implicit Runge-Kutta methods , 1980 .

[4]  Kyle A. Gallivan,et al.  An Algorithm for the Detection and Integration of Highly Oscillatory Ordinary Differential Equations Using a Generalized Unified Modified Divided Difference Representation , 1983 .

[5]  T. E. Hull,et al.  Comparing numerical methods for stiff systems of O.D.E:s , 1975 .

[6]  H. A. Watts,et al.  Software for ordinary differential equations , 1981 .

[7]  G. Dahlquist.  A special stability problem for linear multistep methods , 1963 .

[8]  C. S. Wallace,et al.  General Linear Multistep Methods To Solve Ordinary Differential Equations , 1973, Aust. Comput. J..

[9]  A. Nordsieck,et al.  On Numerical Integration of Ordinary Differential Equations , 1953 .

[10]  L. Petzold.  A description of dassl: a differential/algebraic system solver , 1982 .

[11]  W. H. Enright,et al.  Test Results on Initial Value Methods for Non-Stiff Ordinary Differential Equations , 1976 .

[12]  R. Scheid.  The Accurate Numerical Solution of Highly Oscillatory Ordinary Differential Equations , 1983 .

[13]  John D. Ramsdell,et al.  Estimation of Sparse Jacobian Matrices , 1983 .

[14]  H. A. Watts,et al.  Block Implicit One-Step Methods* , 1969 .

[15]  W. H. Enright.  Optimal Second Derivative Methods for Stiff Systems , 1974 .

[16]  P. Albrecht.  On the order of composite multistep methods for ordinary differential equations , 1978 .

[17]  Richard F. Sincovec,et al.  Software for Nonlinear Partial Differential Equations , 1975, TOMS.

[18]  Michael A. Malcolm,et al.  Computer methods for mathematical computations , 1977 .

[19]  Lawrence F. Shampine,et al.  Type-Insensitive ODE Codes Based on Extrapolation Methods , 1983 .

[20]  W.M.G. van Bokhoven,et al.  Efficient higher order implicit one-step methods for integration of stiff differential equations , 1980 .

[21]  L. F. Shampine,et al.  Type-insensitive ODE codes based on implicit A-stable formulas , 1981 .

[22]  J. H. Verner.  Families of Imbedded Runge–Kutta Methods , 1979 .

[23]  Kyle A. Gallivan,et al.  Automatic methods for highly oscillatory ordinary differential equations , 1982 .

[24]  L. Shampine,et al.  Computer solution of ordinary differential equations : the initial value problem , 1975 .

[25]  A. Sedgwick,et al.  An effective variable-order variable-step adams method. , 1973 .

[26]  H. A. Watts,et al.  Solving Nonstiff Ordinary Differential Equations—The State of the Art , 1976 .

[27]  A. Booth.  Numerical Methods , 1957, Nature.

[28]  P. M. Dew,et al.  Estimating and controlling the global error in Gear's method , 1979 .

[29]  Peter Deuflhard,et al.  Larkin — A Software Package for the Numerical Simulation of LARge Systems Arising in Chemical Reaction KINetics , 1981 .

[30]  Lawrence F. Shampine,et al.  Variable order Runge--Kutta codes , 1979 .

[31]  E. Hairer,et al.  On the stability of semi-implicit methods for ordinary differential equations , 1982 .

[32]  J. Lambert.  Computer Solution of Ordinary Differential Equations , 1976 .

[33]  T. Steihaug,et al.  An attempt to avoid exact Jacobian and nonlinear equations in the numerical solution of stiff differential equations , 1979 .

[34]  M. Powell,et al.  On the Estimation of Sparse Jacobian Matrices , 1974 .

[35]  Charles William Gear.  Numerical Solution of Ordinary Differential Equations: Is There Anything Left to Do? , 1981 .

[36]  Richard C Aiken,et al.  Stiff computation , 1985 .

[37]  Ian Gladwell,et al.  Initial Value Routines in the NAG Library , 1979, TOMS.

[38]  P. Brown,et al.  Matrix-free methods for stiff systems of ODE's , 1986 .

[39]  Gerhard Wanner,et al.  A study of Rosenbrock-type methods of high order , 1981 .

[40]  Sacks-DavisRon,et al.  A review of recent developments in solving ODEs , 1985 .

[41]  Lawrence F. Shampine,et al.  Evaluation of a Test Set for Stiff ODE Solvers , 1981, TOMS.

[42]  M. B. Carver,et al.  On the Use of Sparse Matrix Approximation to the Jacobian in Integrating Large Sets of Ordinary Differential Equations , 1981 .

[43]  A. Hindmarsh.  LSODE and LSODI, two new initial value ordinary differential equation solvers , 1980, SGNM.

[44]  A. Singhal,et al.  Implicit Runge-Kutta formulae for the numerical integration of ODEs , 1980 .

[45]  Lawrence F. Shampine,et al.  Global error estimation for stiff ODEs , 1984 .

[46]  P. Deuflhard,et al.  A semi-implicit mid-point rule for stiff systems of ordinary differential equations , 1983 .

[47]  D. Ellison,et al.  Efficient automatic integration of ordinary differential equations with discontinuities , 1981 .

[48]  R. McCann.  On absolute stability , 1972 .

[49]  P. Henrici.  Discrete Variable Methods in Ordinary Differential Equations , 1962 .

[50]  Alan C. Hindmarsh.  Preliminary documentation of gearbi: solution of ode systems with block iterative treatment of the j , 1976 .

[51]  J. Stoer,et al.  Numerical treatment of ordinary differential equations by extrapolation methods , 1966 .

[52]  L. Petzold.  Automatic Selection of Methods for Solving Stiff and Nonstiff Systems of Ordinary Differential Equations , 1983 .

[53]  F. Chipman.  A-stable Runge-Kutta processes , 1971 .

[54]  Fred T. Krogh,et al.  Asymptotic (h > oo) Absolute Stability for BDFs Applied to Stiff Differential Equations , 1984 .

[55]  Jeff R. Cash,et al.  A Class of Implicit Runge-Kutta Methods for the Numerical Integration of Stiff Ordinary Differential Equations , 1975, JACM.

[56]  Lawrence F. Shampine,et al.  The Art of Writing a Runge-Kutta Code, Part I , 1977 .

[57]  Byron L. Ehle,et al.  High order a-stable methods for the numerical solution of systems of D.E.'s , 1968 .

[58]  W. Enright,et al.  Studies in the numerical-solution of stiff ordinary differential equations. , 1972 .

[59]  Kenneth R. Jackson,et al.  A comparison of two ode codes: gear and episode , 1977 .

[60]  J. M. Varah.  Stiffly Stable Linear Multistep Methods of Extended Order , 1978 .

[61]  Fred T. Krogh,et al.  On Testing a Subroutine for the Numerical Integration of Ordinary Differential Equations , 1973, JACM.

[62]  C. W. Gear,et al.  Numerical initial value problem~ in ordinary differential eqttations , 1971 .

[63]  Robert D. Skeel,et al.  Blended Linear Multistep Methods , 1977, TOMS.

[64]  L. Petzold.  Differential/Algebraic Equations are not ODE's , 1982 .

[65]  D. Arnold,et al.  Computer Solution of Ordinary Differential Equations. , 1981 .

[66]  Peter Piotrowski.  Stability, consistency and convergence of variable K-step methods for numerical integration of large systems of ordinary differential equations , 1969 .

[67]  T. E. Hull,et al.  Comparing Numerical Methods for the Solution of Stiff Systems of ODEs Arising in Chemistry , 1976 .

[68]  Hans J. Stetter.  Economical Global Error Estimation , 1974 .

[69]  H. H. Rosenbrock,et al.  Some general implicit processes for the numerical solution of differential equations , 1963, Comput. J..

[70]  Wayne H. Enright,et al.  Improving the Efficiency of Matrix Operations in the Numerical Solution of Stiff Ordinary Differential Equations , 1978, TOMS.

[71]  A. Hindmarsh,et al.  GEAR: ORDINARY DIFFERENTIAL EQUATION SYSTEM SOLVER. , 1971 .

[72]  I. Gladwell,et al.  Computational techniques for ordinary differential equations , 1982 .

[73]  George F. Corliss,et al.  Solving Ordinary Differential Equations Using Taylor Series , 1982, TOMS.

[74]  Hans J. Stetter.  Interpolation and Error Estimation in Adams $PC$-Codes , 1979 .

[75]  Kevin Burrage,et al.  A special family of Runge-Kutta methods for solving stiff differential equations , 1978 .

[76]  P. Deuflhard.  Order and stepsize control in extrapolation methods , 1983 .

[77]  Eldon Hansen,et al.  Cyclic composite multistep predictor-corrector methods , 1969, ACM '69.

[78]  L. F. Shampine,et al.  Evaluation of implicit formulas for the solution of ODEs , 1979 .

[79]  G. Gupta,et al.  Some new high-order multistep formulae for solving stiff equations , 1976 .

[80]  E. Hairer,et al.  Order stars and stability theorems , 1978 .

[81]  C F Curtiss,et al.  Integration of Stiff Equations. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[82]  David Barton,et al.  On Taylor Series and Stiff Equations , 1980, TOMS.

[83]  L. Shampine.  Implementation of Implicit Formulas for the Solution of ODEs , 1980 .

[84]  Germund Dahlquist.  On the control of the global error in stiff initial value problems , 1982 .

[85]  Alan C. Hindmarsh,et al.  A Polyalgorithm for the Numerical Solution of Ordinary Differential Equations , 1975, TOMS.

[86]  B. Lindberg.  On smoothing and extrapolation for the trapezoidal rule , 1971 .

[87]  L. Petzold.  An Efficient Numerical Method for Highly Oscillatory Ordinary Differential Equations , 1978 .

[88]  P. Oliver,et al.  A Family of Linear Multistep Methods for the Solution of Stiff and Non-stiff ODEs , 1982 .

[89]  Antony King-Yin Kong.  A Search for Better Linear Multistep Methods for Stiff Problems , 1977 .

[90]  H. J. Stetter,et al.  Global error estimation in ODE-solvers , 1978 .

[91]  John C. Butcher,et al.  Integration processes based on Radau quadrature formulas , 1964 .

[92]  Theodore A. Bickart,et al.  High order stiffly stable composite multistep methods for numerical integration of stiff differential equations , 1973 .

[93]  John C. Strikwerda,et al.  Iterative Methods for the Numerical Solution of Second Order Elliptic Equations with Large First Order Terms , 1980 .

[94]  T. Chan,et al.  Nonlinearly Preconditioned Krylov Subspace Methods for Discrete Newton Algorithms , 1984 .

[95]  H. A. Watts.  Starting step size for an ODE solver , 1983 .

[96]  D. N. P. Murthy,et al.  Two classes of internally s-stable generalized runge-kutta processes which remain consistent with an inaccurate jacobian , 1982 .

[97]  J. Cash,et al.  Mono-implicit Runge—Kutta Formulae for the Numerical Integration of Stiff Differential Systems , 1982 .

[98]  Stig Skelboe,et al.  Backward differentiation formulas with extended regions of absolute stability , 1981 .

[99]  Ron Sacks-Davis,et al.  Fixed Leading Coefficient Implementation of SD-Formulas for Stiff ODEs , 1980, TOMS.

[100]  A. H. Sherman,et al.  GEARS: a package for the solution of sparse, stiff ordinary differential equations. [In Fortran] , 1980 .

[101]  M. B. Carver.  The Choice of Algorithms in Automated Method of Lines Solution of Partial Differential Equations , 1976 .

[102]  Åke Björck,et al.  Numerical Methods , 1995, Handbook of Marine Craft Hydrodynamics and Motion Control.

[103]  C. W. Geart,et al.  THE EFFECT OF VARIABLE MESH SIZE ON THE STABILITY OF MULTISTEP METHODS , 1974 .

[104]  A. Sayfy,et al.  Semiexplicit A -Stable Runge-Kutta Methods , 1979 .

[105]  David S. Watkins,et al.  The Numerical Solution of Separably Stiff Systems by Precise Partitioning , 1983, TOMS.

[106]  C. W. Gear,et al.  Algorithm 407: DIFSUB for solution of ordinary differential equations [D2] , 1971, Commun. ACM.

[107]  Patrick W. Gaffney,et al.  A Performance Evaluation of Some FORTRAN Subroutines for the Solution of Stiff Oscillatory Ordinary Differential Equations , 1984, TOMS.

[108]  C. W. Gear,et al.  Iterative Solution of Linear Equations in ODE Codes , 1983 .

[109]  R. Sacks-Davis,et al.  A New Class of Cyclic Multistep Formulae for Stiff Systems , 1983 .

[110]  C. W. Gear,et al.  The automatic integration of ordinary differential equations , 1971, Commun. ACM.

[111]  William B. Gragg,et al.  On Extrapolation Algorithms for Ordinary Initial Value Problems , 1965 .

[112]  J. Lambert.  Computational Methods in Ordinary Differential Equations , 1973 .

[113]  Thomas F. Coleman,et al.  Software for estimating sparse Jacobian matrices , 1984, ACM Trans. Math. Softw..

[114]  Lawrence F. Shampine,et al.  The art of writing a Runge-Kutta code. II. , 1979 .

[115]  Hans J. Stetter.  Global Error Estimation in Adams PC-Codes , 1979, TOMS.

[116]  P. A. Fox,et al.  DESUB: INTEGRATION OF A FIRST-ORDER SYSTEM OF ORDINARY DIFFERENTIAL EQUATIONS , 1971 .

[117]  Charles William Gear.  Multirate methods for ordinary differential equations , 1974 .

[118]  H. A. Watts,et al.  DEPAC - design of a user oriented package of ODE solvers , 1980 .

[119]  J. Butcher.  Implicit Runge-Kutta processes , 1964 .

[120]  N. K. Madsen,et al.  Simulation of chemical kinetics transport in the stratosphere , 1974 .

[121]  R. Alexander.  Diagonally implicit runge-kutta methods for stiff odes , 1977 .

[122]  P. Rentrop,et al.  Generalized Runge-Kutta methods of order four with stepsize control for stiff ordinary differential equations , 1979 .

[123]  R. Brayton,et al.  A new efficient algorithm for solving differential-algebraic systems using implicit backward differentiation formulas , 1972 .

[124]  Fred T. Krogh,et al.  Asymptotic (h\rightarrow\infty) Absolute Stability for BDFs Applied to Stiff Differential Equations , 1984, TOMS.

[125]  D. G. Bettis.  Efficient embedded runge — Kutta methods , 1978 .

[126]  T. E. Hull.  The numerical integration of ordinary differential equations , 1968, IFIP Congress.

[127]  C. W. Gear,et al.  ODE METHODS FOR THE SOLUTION OF DIFFERENTIAL/ALGEBRAIC SYSTEMS , 1984 .

[128]  Richard C Aiken.  Proceedings of the International Conference on Stiff Computation, April 12-14, 1982, Park City, Utah. Volume II. , 1982 .

[129]  John C. Butcher,et al.  A Transformed implicit Runge-Kutta Method , 1979, JACM.

[130]  F. Grund.  Forsythe, G. E. / Malcolm, M. A. / Moler, C. B., Computer Methods for Mathematical Computations. Englewood Cliffs, New Jersey 07632. Prentice Hall, Inc., 1977. XI, 259 S , 1979 .

[131]  Mohamed S. Kamel,et al.  Automatic Partitioning of Stiff Systems and Exploiting the Resulting Structure , 1979, TOMS.

[132]  Lawrence F. Shampine,et al.  A User’s View of Solving Stiff Ordinary Differential Equations , 1979 .

[133]  Lawrence F. Shampine.  Stiffness and Nonstiff Differential Equation Solvers, II: Detecting Stiffness with Runge-Kutta Methods , 1977, TOMS.

[134]  P. Zadunaisky.  On the estimation of errors propagated in the numerical integration of ordinary differential equations , 1976 .

[135]  Lawrence F. Shampine,et al.  A type-insensitive ode code based on second derivative formulas , 1981 .

[136]  J. M. Watt.  Numerical Initial Value Problems in Ordinary Differential Equations , 1972 .

[137]  Ron Sacks-Davis,et al.  An Alternative Implementation of Variable Step-Size Multistep Formulas for Stiff ODEs , 1980, TOMS.

[138]  A. R. Gourlay,et al.  An Implementation of Gear’s Algorithm for CSMP III , 1974 .

[139]  Hindmarsh.  ODE solvers for use with the method of lines. [Brief descriptions of LSODE and LSODI for CDC 7600] , 1980 .

[140]  G. K. Gupta,et al.  A Note About Overhead Costs in ODE Solvers , 1980, TOMS.

[141]  W. Liniger,et al.  A-stable, accurate averaging of multistep methods for stiff differential equations , 1972 .

[142]  G Bjurel,et al.  Survey of stiff ordinary differential equations , 1970 .

[143]  T. Chan,et al.  The Use of Iterative Linear-Equation Solvers in Codes for Large Systems of Stiff IVPs for ODEs , 1986 .

[144]  M. B. Carver.  Efficient integration over discontinuities in ordinary differential equation simulations , 1978 .

[145]  Joel M. Tendler,et al.  A stiffly stable integration process using cyclic composite methods , 1978, TOMS.

[146]  G. Gupta,et al.  Description and Evaluation of a Stiff ODE Code DSTIFF , 1985 .

[147]  C. W. Gear,et al.  Simultaneous Numerical Solution of Differential-Algebraic Equations , 1971 .

[148]  Theodore A. Bickart,et al.  A-Stable Composite Multistep Methods , 1973, JACM.

[149]  Lawrence F. Shampine,et al.  Global Error Estimates for Ordinary Differential Equations , 1976, TOMS.

[150]  T. E. Hull,et al.  Comparing Numerical Methods for Ordinary Differential Equations , 1972 .

[151]  Theodore A. Bickart,et al.  An Efficient Solution Process for Implicit Runge–Kutta Methods , 1977 .

[152]  Frank de Hoog,et al.  A Class of A-Stable Advanced Multistep Methods , 1974 .

[153]  Ole Østerby,et al.  Solving Ordinary Differential Equations with Discontinuities , 1984, TOMS.

[154]  Åke Björck,et al.  A blockQR algorithm for partitioning stiff differential systems , 1983 .

[155]  J. Verner.  Explicit Runge–Kutta Methods with Estimates of the Local Truncation Error , 1978 .

[156]  C. S. Wallace,et al.  Some new multistep methods for solving ordinary differential equations , 1975 .

[157]  H. A. Watts,et al.  A-stable block implicit one-step methods , 1972 .

[158]  J. F. Andrus,et al.  Numerical Solution of Systems of Ordinary Differential Equations Separated into Subsystems , 1979 .

[159]  J. Barkley Rosser,et al.  A Runge-Kutta for all Seasons , 1967 .

[160]  J. Lambert,et al.  Multistep Methods with Variable Matrix Coefficients , 1972 .