Approximation and inference methods for stochastic biochemical kinetics—a tutorial review

Stochastic fluctuations of molecule numbers are ubiquitous in biological systems. Important examples include gene expression and enzymatic processes in living cells. Such systems are typically modelled as chemical reaction networks whose dynamics are governed by the chemical master equation. Despite its simple structure, no analytic solutions to the chemical master equation are known for most systems. Moreover, stochastic simulations are computationally expensive, making systematic analysis and statistical inference a challenging task. Consequently, significant effort has been spent in recent decades on the development of efficient approximation and inference methods. This article gives an introduction to basic modelling concepts as well as an overview of state of the art methods. First, we motivate and introduce deterministic and stochastic methods for modelling chemical networks, and give an overview of simulation and exact solution methods. Next, we discuss several approximation methods, including the chemical Langevin equation, the system size expansion, moment closure approximations, time-scale separation approximations and hybrid methods. We discuss their various properties and review recent advances and remaining challenges for these methods. We present a comparison of several of these methods by means of a numerical case study and highlight some of their respective advantages and disadvantages. Finally, we discuss the problem of inference from experimental data in the Bayesian framework and review recent methods developed the literature. In summary, this review gives a D Schnoerr et al Approximation and inference methods for stochastic biochemical kinetics—a tutorial review Printed in the UK 093001 JPHAC5 © 2017 IOP Publishing Ltd 50 J. Phys. A: Math. Theor.

[1]  J. E. Moyal Stochastic Processes and Statistical Physics , 1949 .

[2]  G. Sanguinetti,et al.  Learning and Designing Stochastic Processes from Logical Constraints , 2013, QEST.

[3]  E. O’Shea,et al.  Noise in protein expression scales with natural protein abundance , 2006, Nature Genetics.

[4]  M. Peter,et al.  Scalable inference of heterogeneous reaction kinetics from pooled single-cell recordings , 2013, Nature Methods.

[5]  G. Verghese,et al.  Enhanced identification and exploitation of time scales for model reduction in stochastic chemical kinetics. , 2008, The Journal of chemical physics.

[6]  Andreas Ruttor,et al.  Approximate parameter inference in a stochastic reaction-diffusion model , 2010, AISTATS.

[7]  Todd K. Leen,et al.  Perturbation theory for stochastic learning dynamics , 2011, The 2011 International Joint Conference on Neural Networks.

[8]  J.P. Hespanha,et al.  Lognormal Moment Closures for Biochemical Reactions , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[9]  M. Delbrück Statistical Fluctuations in Autocatalytic Reactions , 1940 .

[10]  Heinz Koeppl,et al.  Jump-Diffusion Approximation of Stochastic Reaction Dynamics: Error Bounds and Algorithms , 2014, Multiscale Model. Simul..

[11]  Guido Sanguinetti,et al.  The complex chemical Langevin equation. , 2014, The Journal of chemical physics.

[12]  D. Gillespie The chemical Langevin equation , 2000 .

[13]  Guido Sanguinetti,et al.  Approximate inference in latent Gaussian-Markov models from continuous time observations , 2013, NIPS.

[14]  Michael P H Stumpf,et al.  Multivariate moment closure techniques for stochastic kinetic models. , 2015, The Journal of chemical physics.

[15]  Andrew Golightly,et al.  Bayesian inference for hybrid discrete-continuous stochastic kinetic models , 2014, 1402.6602.

[16]  Peter G. Hufton,et al.  Intrinsic noise in systems with switching environments. , 2015, Physical review. E.

[17]  Kevin R. Sanft,et al.  Legitimacy of the stochastic Michaelis-Menten approximation. , 2011, IET systems biology.

[18]  M. Doi,et al.  Second quantization representation for classical many-particle system , 2001 .

[19]  Ilya Nemenman,et al.  Adiabatic coarse-graining and simulations of stochastic biochemical networks , 2009, Proceedings of the National Academy of Sciences.

[20]  Natasha A. Neogi,et al.  Dynamic Partitioning of Large Discrete Event Biological Systems for Hybrid Simulation and Analysis , 2004, HSCC.

[21]  Chetan D. Pahlajani,et al.  Stochastic reduction method for biological chemical kinetics using time-scale separation. , 2011, Journal of theoretical biology.

[22]  D. Gillespie,et al.  A diffusional bimolecular propensity function. , 2009, The Journal of chemical physics.

[23]  J. Goutsias,et al.  Numerical Integration of the Master Equation in Some Models of Stochastic Epidemiology , 2012, PloS one.

[24]  T. Faniran Numerical Solution of Stochastic Differential Equations , 2015 .

[25]  R. Jackson,et al.  General mass action kinetics , 1972 .

[26]  Jae Kyoung Kim,et al.  The relationship between stochastic and deterministic quasi-steady state approximations , 2015, BMC Systems Biology.

[27]  Marco Beccuti,et al.  Analysis of Petri Net Models through Stochastic Differential Equations , 2014, Petri Nets.

[28]  E Weinan,et al.  Nested stochastic simulation algorithms for chemical kinetic systems with multiple time scales , 2007, J. Comput. Phys..

[29]  M. Scott,et al.  Non-linear corrections to the time-covariance function derived from a multi-state chemical master equation. , 2012, IET systems biology.

[30]  Philipp Thomas,et al.  Communication: limitations of the stochastic quasi-steady-state approximation in open biochemical reaction networks. , 2011, The Journal of chemical physics.

[31]  S. Isaacson Relationship between the reaction–diffusion master equation and particle tracking models , 2008 .

[32]  Haluk Resat,et al.  Multinomial tau-leaping method for stochastic kinetic simulations. , 2007, The Journal of chemical physics.

[33]  G. Briggs,et al.  A Note on the Kinetics of Enzyme Action. , 1925, The Biochemical journal.

[34]  J. Hespanha Moment closure for biochemical networks , 2008, 2008 3rd International Symposium on Communications, Control and Signal Processing.

[35]  Ohira,et al.  Master-equation approach to stochastic neurodynamics. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[36]  Marco Beccuti,et al.  Approximate analysis of biological systems by hybrid switching jump diffusion , 2014, Theor. Comput. Sci..

[37]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[38]  Giancarlo Mauri,et al.  The Interplay of Intrinsic and Extrinsic Bounded Noises in Biomolecular Networks , 2012, PloS one.

[39]  Roger B Sidje,et al.  Understanding the finite state projection and related methods for solving the chemical master equation , 2016, Physical biology.

[40]  Hong Li,et al.  Algorithms and Software for Stochastic Simulation of Biochemical Reacting Systems , 2008, Biotechnology progress.

[41]  R. F. Pawula,et al.  Approximation of the Linear Boltzmann Equation by the Fokker-Planck Equation , 1967 .

[42]  Jürgen Pahle,et al.  Biochemical simulations: stochastic, approximate stochastic and hybrid approaches , 2008, Briefings Bioinform..

[43]  J. Timmer,et al.  Noisy signaling through promoter logic gates. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[44]  Corrections to the law of mass action and properties of the asymptotic t = ' state for reversible diffusion-limited reactions , 2005, cond-mat/0503198.

[45]  Ramon Grima,et al.  Analytical approximations for spatial stochastic gene expression in single cells and tissues , 2016, Journal of The Royal Society Interface.

[46]  Sheng Wu,et al.  The time dependent propensity function for acceleration of spatial stochastic simulation of reaction-diffusion systems , 2014, J. Comput. Phys..

[47]  M. Feinberg The existence and uniqueness of steady states for a class of chemical reaction networks , 1995 .

[48]  S. Frühwirth-Schnatter Data Augmentation and Dynamic Linear Models , 1994 .

[49]  Guido Sanguinetti,et al.  Variational inference for Markov jump processes , 2007, NIPS.

[50]  Kevin Burrage,et al.  Stochastic approaches for modelling in vivo reactions , 2004, Comput. Biol. Chem..

[51]  Philipp Thomas,et al.  Rigorous elimination of fast stochastic variables from the linear noise approximation using projection operators. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  Sean R. Anderson,et al.  Repelled from the wound, or randomly dispersed? Reverse migration behaviour of neutrophils characterized by dynamic modelling , 2012, Journal of The Royal Society Interface.

[53]  J. Elf,et al.  Lost in presumption: stochastic reactions in spatial models , 2012, Nature Methods.

[54]  Radek Erban,et al.  Tensor methods for parameter estimation and bifurcation analysis of stochastic reaction networks , 2015, Journal of The Royal Society Interface.

[55]  Aleksandar Donev,et al.  A First-Passage Kinetic Monte Carlo algorithm for complex diffusion-reaction systems , 2009, J. Comput. Phys..

[56]  Ruth J. Williams,et al.  Correlation resonance generated by coupled enzymatic processing. , 2010, Biophysical journal.

[57]  Y. Wong,et al.  Positivity preserving chemical Langevin equations , 2008 .

[58]  Linda R. Petzold,et al.  Stochastic modelling of gene regulatory networks , 2005 .

[59]  J Halloy,et al.  Deterministic Versus Stochastic Models for Circadian Rhythms , 2002, Journal of biological physics.

[60]  Anthony F. Bartholomay,et al.  Stochastic models for chemical reactions: I. Theory of the unimolecular reaction process , 1958 .

[61]  Erika Cule,et al.  ABC-SysBio—approximate Bayesian computation in Python with GPU support , 2010, Bioinform..

[62]  István Simon,et al.  Self-regulating genes. Exact steady state solution by using Poisson representation , 2013, 1312.3919.

[63]  J. Hillston,et al.  Stochastic properties of the plant circadian clock , 2012, Journal of The Royal Society Interface.

[64]  D. Wilkinson,et al.  Bayesian Inference for Stochastic Kinetic Models Using a Diffusion Approximation , 2005, Biometrics.

[65]  C. Gillespie Moment-closure approximations for mass-action models. , 2009, IET systems biology.

[66]  Angelo Vulpiani,et al.  Coarse graining of master equations with fast and slow states. , 2008, The Journal of chemical physics.

[67]  Desmond J. Higham,et al.  An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations , 2001, SIAM Rev..

[68]  Cosmin Safta,et al.  Hybrid discrete/continuum algorithms for stochastic reaction networks , 2015, J. Comput. Phys..

[69]  Fabian J. Theis,et al.  CERENA: ChEmical REaction Network Analyzer—A Toolbox for the Simulation and Analysis of Stochastic Chemical Kinetics , 2016, PloS one.

[70]  David F Anderson,et al.  A modified next reaction method for simulating chemical systems with time dependent propensities and delays. , 2007, The Journal of chemical physics.

[71]  C Jayaprakash,et al.  Mixed Poisson distributions in exact solutions of stochastic autoregulation models. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[72]  Erwin Frey,et al.  Master equations and the theory of stochastic path integrals , 2016, Reports on progress in physics. Physical Society.

[73]  Guido Sanguinetti,et al.  Validity conditions for moment closure approximations in stochastic chemical kinetics. , 2014, The Journal of chemical physics.

[74]  Alain Destexhe,et al.  A Master Equation Formalism for Macroscopic Modeling of Asynchronous Irregular Activity States , 2009, Neural Computation.

[75]  Hamid Bolouri,et al.  Dizzy: Stochastic Simulation of Large-scale Genetic Regulatory Networks , 2005, J. Bioinform. Comput. Biol..

[76]  M. Keeling,et al.  On methods for studying stochastic disease dynamics , 2008, Journal of The Royal Society Interface.

[77]  Timo R. Maarleveld,et al.  StochPy: A Comprehensive, User-Friendly Tool for Simulating Stochastic Biological Processes , 2013, PloS one.

[78]  Niraj Kumar,et al.  Exact distributions for stochastic gene expression models with bursting and feedback. , 2014, Physical review letters.

[79]  Guido Sanguinetti,et al.  Unbiased Bayesian inference for population Markov jump processes via random truncations , 2015, Stat. Comput..

[80]  J. Tóth,et al.  A full stochastic description of the Michaelis-Menten reaction for small systems. , 1977, Acta biochimica et biophysica; Academiae Scientiarum Hungaricae.

[81]  Guido Sanguinetti,et al.  Hybrid regulatory models: a statistically tractable approach to model regulatory network dynamics , 2013, Bioinform..

[82]  Eric Vanden-Eijnden,et al.  Nested stochastic simulation algorithm for chemical kinetic systems with disparate rates. , 2005, The Journal of chemical physics.

[83]  Guido Sanguinetti,et al.  Expectation propagation for continuous time stochastic processes , 2015, 1512.06098.

[84]  D. Gillespie Exact Stochastic Simulation of Coupled Chemical Reactions , 1977 .

[85]  R. Heinrich,et al.  The Regulation of Cellular Systems , 1996, Springer US.

[86]  Andrew J. Black,et al.  Stochasticity in staged models of epidemics: quantifying the dynamics of whooping cough , 2010, Journal of The Royal Society Interface.

[87]  Wolf Verena,et al.  Model Reconstruction for Moment-Based Stochastic Chemical Kinetics , 2014, ACM Trans. Model. Comput. Simul..

[88]  Paulette Clancy,et al.  A "partitioned leaping" approach for multiscale modeling of chemical reaction dynamics. , 2006, The Journal of chemical physics.

[89]  Eric Bonabeau,et al.  Agent-based modeling: Methods and techniques for simulating human systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[90]  Wolfgang Weidlich,et al.  Sociodynamics: a Systematic Approach to Mathematical Modelling in the Social Sciences , 2000 .

[91]  C. Rao,et al.  Stochastic chemical kinetics and the quasi-steady-state assumption: Application to the Gillespie algorithm , 2003 .

[92]  C. Gardiner,et al.  The poisson representation. I. A new technique for chemical master equations , 1977 .

[93]  Ramon Grima,et al.  Model reduction for stochastic chemical systems with abundant species. , 2015, The Journal of chemical physics.

[94]  Xiaodong Cai,et al.  K-leap method for accelerating stochastic simulation of coupled chemical reactions. , 2007, The Journal of chemical physics.

[95]  M. Elowitz,et al.  Functional roles for noise in genetic circuits , 2010, Nature.

[96]  Darren J. Wilkinson,et al.  Bayesian inference for a discretely observed stochastic kinetic model , 2008, Stat. Comput..

[97]  A. Kierzek,et al.  Bridging the gap between stochastic and deterministic regimes in the kinetic simulations of the biochemical reaction networks. , 2004, Biophysical journal.

[98]  Marc Benayoun,et al.  Emergent Oscillations in Networks of Stochastic Spiking Neurons , 2011, PloS one.

[99]  Masaru Tomita,et al.  A multi-algorithm, multi-timescale method for cell simulation , 2004, Bioinform..

[100]  A. Duncan,et al.  Noise-induced multistability in chemical systems: Discrete versus continuum modeling. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[101]  LUKASZ SZPRUCH,et al.  Comparing Hitting Time Behavior of Markov Jump Processes and Their Diffusion Approximations , 2010, Multiscale Model. Simul..

[102]  Wolfgang Weidlich,et al.  Physics and social science — The approach of synergetics , 1991 .

[103]  R. Grima,et al.  Intrinsic Noise Analyzer: A Software Package for the Exploration of Stochastic Biochemical Kinetics Using the System Size Expansion , 2012, PloS one.

[104]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[105]  K. Ishida,et al.  Stochastic Model for Bimolecular Reaction , 1964 .

[106]  Joseph D Challenger,et al.  Synchronization of stochastic oscillators in biochemical systems. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[107]  David F. Anderson,et al.  Product-Form Stationary Distributions for Deficiency Zero Chemical Reaction Networks , 2008, Bulletin of mathematical biology.

[108]  Linda R Petzold,et al.  Adaptive explicit-implicit tau-leaping method with automatic tau selection. , 2007, The Journal of chemical physics.

[109]  R. Law,et al.  A Jump-Growth Model for Predator–Prey Dynamics: Derivation and Application to Marine Ecosystems , 2008, Bulletin of mathematical biology.

[110]  P. Staff,et al.  Stochastic Approach to First-Order Chemical Reaction Kinetics , 1966 .

[111]  Radek Erban,et al.  Hybrid framework for the simulation of stochastic chemical kinetics , 2016, J. Comput. Phys..

[112]  João P. Hespanha,et al.  Equilibrium distributions of simple biochemical reaction systems for time-scale separation in stochastic reaction networks , 2014, Journal of The Royal Society Interface.

[113]  J. Rawlings,et al.  Approximate simulation of coupled fast and slow reactions for stochastic chemical kinetics , 2002 .

[114]  M. Thattai,et al.  Intrinsic noise in gene regulatory networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[115]  M. Scott,et al.  Approximating intrinsic noise in continuous multispecies models , 2011, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[116]  Darren J. Wilkinson,et al.  Moment closure based parameter inference of stochastic kinetic models , 2013, Stat. Comput..

[117]  P. T. Wolde,et al.  Simulating biochemical networks at the particle level and in time and space: Green's function reaction dynamics. , 2005 .

[118]  R. Grima,et al.  How reliable is the linear noise approximation of gene regulatory networks? , 2013, BMC Genomics.

[119]  Andreas Hellander,et al.  Perspective: Stochastic algorithms for chemical kinetics. , 2013, The Journal of chemical physics.

[120]  E. O’Shea,et al.  Living with noisy genes: how cells function reliably with inherent variability in gene expression. , 2007, Annual review of biophysics and biomolecular structure.

[121]  S. Jonathan Chapman,et al.  Analysis of Brownian Dynamics Simulations of Reversible Bimolecular Reactions , 2010, SIAM J. Appl. Math..

[122]  L. Petzold,et al.  Reaction-diffusion master equation in the microscopic limit. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[123]  Noel A Cressie,et al.  Statistics for Spatio-Temporal Data , 2011 .

[124]  P R Taylor,et al.  Deriving appropriate boundary conditions, and accelerating position-jump simulations, of diffusion using non-local jumping , 2014, Physical biology.

[125]  Paul D. W. Kirk,et al.  MEANS: python package for Moment Expansion Approximation, iNference and Simulation , 2016, Bioinform..

[126]  Louise Dyson,et al.  Noise-induced bistable states and their mean switching time in foraging colonies. , 2013, Physical review letters.

[127]  D. Fanelli,et al.  Analytical study of non Gaussian fluctuations in a stochastic scheme of autocatalytic reactions , 2011, 1104.5668.

[128]  Guido Sanguinetti,et al.  Reconstructing transcription factor activities in hierarchical transcription network motifs , 2011, Bioinform..

[129]  D G Vlachos,et al.  Overcoming stiffness in stochastic simulation stemming from partial equilibrium: a multiscale Monte Carlo algorithm. , 2005, The Journal of chemical physics.

[130]  J. Bowen,et al.  Singular perturbation refinement to quasi-steady state approximation in chemical kinetics , 1963 .

[131]  Ankit Gupta,et al.  Adaptive hybrid simulations for multiscale stochastic reaction networks. , 2014, The Journal of chemical physics.

[132]  K. Burrage,et al.  Numerical methods for strong solutions of stochastic differential equations: an overview , 2004, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[133]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[134]  Donald A. McQuarrie,et al.  Kinetics of Small Systems. I , 1963 .

[135]  M. S. Bartlett,et al.  Some Evolutionary Stochastic Processes , 1949 .

[136]  Yiannis N. Kaznessis,et al.  A closure scheme for chemical master equations , 2013, Proceedings of the National Academy of Sciences.

[137]  Piet Van Mieghem,et al.  Epidemic processes in complex networks , 2014, ArXiv.

[138]  Ioannis G Kevrekidis,et al.  A constrained approach to multiscale stochastic simulation of chemically reacting systems. , 2011, The Journal of chemical physics.

[139]  Radek Erban,et al.  Error Analysis of Diffusion Approximation Methods for Multiscale Systems in Reaction Kinetics , 2014, SIAM J. Sci. Comput..

[140]  L. A. Segel,et al.  The Quasi-Steady-State Assumption: A Case Study in Perturbation , 1989, SIAM Rev..

[141]  Hong Qian,et al.  Grand canonical Markov model: a stochastic theory for open nonequilibrium biochemical networks. , 2006, The Journal of chemical physics.

[142]  Julien F. Ollivier,et al.  Colored extrinsic fluctuations and stochastic gene expression , 2008, Molecular systems biology.

[143]  Tobias Jahnke,et al.  Error Bound for Piecewise Deterministic Processes Modeling Stochastic Reaction Systems , 2012, Multiscale Model. Simul..

[144]  D. Gillespie A General Method for Numerically Simulating the Stochastic Time Evolution of Coupled Chemical Reactions , 1976 .

[145]  R. Grima,et al.  An effective rate equation approach to reaction kinetics in small volumes: theory and application to biochemical reactions in nonequilibrium steady-state conditions. , 2010, The Journal of chemical physics.

[146]  Michael P H Stumpf,et al.  A general moment expansion method for stochastic kinetic models. , 2013, The Journal of chemical physics.

[147]  A. J. McKane,et al.  Stochastic models of evolution in genetics, ecology and linguistics , 2007, cond-mat/0703478.

[148]  M. Khammash,et al.  The finite state projection algorithm for the solution of the chemical master equation. , 2006, The Journal of chemical physics.

[149]  Muruhan Rathinam,et al.  Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method , 2003 .

[150]  Ovidiu Radulescu,et al.  Hybrid stochastic simplifications for multiscale gene networks , 2009, BMC Systems Biology.

[151]  J. Timmer,et al.  Signatures of nonlinearity in single cell noise-induced oscillations. , 2013, Journal of theoretical biology.

[152]  Grégoire Nicolis,et al.  Self-Organization in nonequilibrium systems , 1977 .

[153]  Axel Kowald,et al.  Systems Biology in Practice: Concepts, Implementation and Application , 2005 .

[154]  Duccio Fanelli,et al.  Enhanced stochastic oscillations in autocatalytic reactions. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[155]  Paul Fearnhead,et al.  Inference for reaction networks using the linear noise approximation , 2012, Biometrics.

[156]  Peter Whittle,et al.  Systems in stochastic equilibrium , 1986 .

[157]  Ingemar Nåsell,et al.  An extension of the moment closure method. , 2003, Theoretical population biology.

[158]  H. El-Samad,et al.  A rigorous framework for multiscale simulation of stochastic cellular networks. , 2009, The Journal of chemical physics.

[159]  J. Lygeros,et al.  Moment-based inference predicts bimodality in transient gene expression , 2012, Proceedings of the National Academy of Sciences.

[160]  Malbor Asllani,et al.  The linear noise approximation for reaction-diffusion systems on networks , 2013, 1305.7318.

[161]  Darren J. Wilkinson,et al.  CaliBayes and BASIS: integrated tools for the calibration, simulation and storage of biological simulation models , 2010, Briefings Bioinform..

[162]  Philipp Thomas,et al.  Stochastic Simulation of Biomolecular Networks in Dynamic Environments , 2015, PLoS Comput. Biol..

[163]  N. Popović,et al.  Phenotypic switching in gene regulatory networks , 2014, Proceedings of the National Academy of Sciences.

[164]  Tobias Jahnke,et al.  On Reduced Models for the Chemical Master Equation , 2011, Multiscale Model. Simul..

[165]  Yiannis Kaznessis,et al.  Accurate hybrid stochastic simulation of a system of coupled chemical or biochemical reactions. , 2005, The Journal of chemical physics.

[166]  Deena R. Schmidt,et al.  Steady-state fluctuations of a genetic feedback loop: an exact solution. , 2012, The Journal of chemical physics.

[167]  Awad H. Al-Mohy,et al.  Computing the Action of the Matrix Exponential, with an Application to Exponential Integrators , 2011, SIAM J. Sci. Comput..

[168]  D. Sherrington Stochastic Processes in Physics and Chemistry , 1983 .

[169]  C. Pesce,et al.  Regulated cell-to-cell variation in a cell-fate decision system , 2005, Nature.

[170]  Saswati Dana,et al.  Physically consistent simulation of mesoscale chemical kinetics: The non-negative FIS-α method , 2011, J. Comput. Phys..

[171]  P. Whittle On the Use of the Normal Approximation in the Treatment of Stochastic Processes , 1957 .

[172]  M J Keeling,et al.  Multiplicative moments and measures of persistence in ecology. , 2000, Journal of theoretical biology.

[173]  Ovidiu Radulescu,et al.  Convergence of stochastic gene networks to hybrid piecewise deterministic processes , 2011, 1101.1431.

[174]  W. Huisinga,et al.  Solving the chemical master equation for monomolecular reaction systems analytically , 2006, Journal of mathematical biology.

[175]  J. Goutsias Quasiequilibrium approximation of fast reaction kinetics in stochastic biochemical systems. , 2005, The Journal of chemical physics.

[176]  Guido Sanguinetti,et al.  Efficient Stochastic Simulation of Systems with Multiple Time Scales via Statistical Abstraction , 2015, CMSB.

[177]  R. Erban,et al.  Stochastic modelling of reaction–diffusion processes: algorithms for bimolecular reactions , 2009, Physical biology.

[178]  Mark A. Girolami,et al.  Markov chain Monte Carlo inference for Markov jump processes via the linear noise approximation , 2012, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[179]  Abhyudai Singh,et al.  Conditional Moment Closure Schemes for Studying Stochastic Dynamics of Genetic Circuits , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[180]  Roland Eils,et al.  General Stochastic Hybrid Method for the Simulation of Chemical Reaction Processes in Cells , 2004, CMSB.

[181]  Carsten Marr,et al.  A geometric analysis of fast-slow models for stochastic gene expression , 2015, Journal of Mathematical Biology.

[182]  T. Kurtz,et al.  Separation of time-scales and model reduction for stochastic reaction networks. , 2010, 1011.1672.

[183]  Philipp Thomas,et al.  The slow-scale linear noise approximation: an accurate, reduced stochastic description of biochemical networks under timescale separation conditions , 2012, BMC Systems Biology.

[184]  Michael A. Buice,et al.  Systematic Fluctuation Expansion for Neural Network Activity Equations , 2009, Neural Computation.

[185]  A. McKane,et al.  Amplified Biochemical Oscillations in Cellular Systems , 2006, q-bio/0604001.

[186]  D. Gillespie A rigorous derivation of the chemical master equation , 1992 .

[187]  S. Isaacson A convergent reaction-diffusion master equation. , 2012, Journal of Chemical Physics.

[188]  Fabian J Theis,et al.  Method of conditional moments (MCM) for the Chemical Master Equation , 2013, Journal of Mathematical Biology.

[189]  Todd K. Leen,et al.  Stochastic Perturbation Methods for Spike-Timing-Dependent Plasticity , 2012, Neural Computation.

[190]  João Pedro Hespanha,et al.  Approximate Moment Dynamics for Chemically Reacting Systems , 2011, IEEE Transactions on Automatic Control.

[191]  Cleve B. Moler,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later , 1978, SIAM Rev..

[192]  Philipp Thomas,et al.  Computation of biochemical pathway fluctuations beyond the linear noise approximation using iNA , 2012, 2012 IEEE International Conference on Bioinformatics and Biomedicine.

[193]  Tianhai Tian,et al.  A multi-scaled approach for simulating chemical reaction systems. , 2004, Progress in biophysics and molecular biology.

[194]  N. Kampen The equilibrium distribution of a chemical mixture , 1976 .

[195]  P J Staff A stochastic development of the reversible Michaelis-Menten mechanism. , 1970, Journal of theoretical biology.

[196]  Guido Sanguinetti,et al.  Comparison of different moment-closure approximations for stochastic chemical kinetics. , 2015, The Journal of chemical physics.

[197]  Leo A. Goodman,et al.  Population Growth of the Sexes , 1953 .

[198]  Per Lötstedt,et al.  Hybrid method for the chemical master equation , 2007 .

[199]  Visakan Kadirkamanathan,et al.  Parameter estimation and inference for stochastic reaction-diffusion systems: application to morphogenesis in D. melanogaster , 2009, BMC Systems Biology.

[200]  P. Swain,et al.  Stochastic Gene Expression in a Single Cell , 2002, Science.

[201]  J. Goutsias,et al.  Markovian dynamics on complex reaction networks , 2012, 1205.5524.

[202]  M. Doi Stochastic theory of diffusion-controlled reaction , 1976 .

[203]  Vahid Shahrezaei,et al.  Analytical distributions for stochastic gene expression , 2008, Proceedings of the National Academy of Sciences.

[204]  Chin-Kun Hu,et al.  Nonequilibrium Lyapunov function and a fluctuation relation for stochastic systems: Poisson-representation approach. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[205]  Philipp Thomas,et al.  Distribution Approximations for the Chemical Master Equation: Comparison of the Method of Moments and the System Size Expansion , 2015, 1509.09104.

[206]  Andreas Ruttor,et al.  Efficient statistical inference for stochastic reaction processes. , 2009, Physical review letters.

[207]  Samuel A. Isaacson,et al.  The Reaction-Diffusion Master Equation as an Asymptotic Approximation of Diffusion to a Small Target , 2009, SIAM J. Appl. Math..

[208]  Yiannis N Kaznessis,et al.  An equation-free probabilistic steady-state approximation: dynamic application to the stochastic simulation of biochemical reaction networks. , 2005, The Journal of chemical physics.

[209]  Van Kampen,et al.  The Expansion of the Master Equation , 2007 .

[210]  Sheng Wu,et al.  StochKit2: software for discrete stochastic simulation of biochemical systems with events , 2011, Bioinform..

[211]  K. Zygalakis,et al.  Fast stochastic simulation of biochemical reaction systems by alternative formulations of the chemical Langevin equation. , 2010, The Journal of chemical physics.

[212]  Yi-fei Wang,et al.  Efficient binomial leap method for simulating chemical kinetics. , 2007, The Journal of chemical physics.

[213]  Philipp Thomas,et al.  Approximate probability distributions of the master equation. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[214]  C. Bianca,et al.  Evaluation of reaction fluxes in stationary and oscillating far-from-equilibrium biological systems , 2015 .

[215]  T. Kurtz Limit theorems and diffusion approximations for density dependent Markov chains , 1976 .

[216]  Yee Whye Teh,et al.  Fast MCMC sampling for Markov jump processes and extensions , 2012, J. Mach. Learn. Res..

[217]  G. Verghese,et al.  Mass fluctuation kinetics: capturing stochastic effects in systems of chemical reactions through coupled mean-variance computations. , 2007, The Journal of chemical physics.

[218]  Nagiza F. Samatova,et al.  Gene network shaping of inherent noise spectra , 2006 .

[219]  Ramon Grima,et al.  A study of the accuracy of moment-closure approximations for stochastic chemical kinetics. , 2012, The Journal of chemical physics.

[220]  A. McKane,et al.  Stochastic formulation of ecological models and their applications. , 2012, Trends in ecology & evolution.

[221]  Pedro Mendes,et al.  Biochemical fluctuations, optimisation and the linear noise approximation , 2012, BMC Systems Biology.

[222]  Upinder S. Bhalla,et al.  Adaptive stochastic-deterministic chemical kinetic simulations , 2004, Bioinform..

[223]  Brian Munsky,et al.  Reduction and solution of the chemical master equation using time scale separation and finite state projection. , 2006, The Journal of chemical physics.

[224]  Mark A. Stalzer,et al.  Efficient Formulations for Exact Stochastic Simulation of Chemical Systems , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[225]  Darren J Wilkinson,et al.  Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo , 2011, Interface Focus.

[226]  D. Gillespie Approximate accelerated stochastic simulation of chemically reacting systems , 2001 .

[227]  Linda R Petzold,et al.  The slow-scale stochastic simulation algorithm. , 2005, The Journal of chemical physics.

[228]  Philipp Thomas,et al.  Stochastic theory of large-scale enzyme-reaction networks: finite copy number corrections to rate equation models. , 2010, The Journal of chemical physics.

[229]  A. Oudenaarden,et al.  Cellular Decision Making and Biological Noise: From Microbes to Mammals , 2011, Cell.

[230]  P. Swain,et al.  Intrinsic and extrinsic contributions to stochasticity in gene expression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[231]  K. Burrage,et al.  Binomial leap methods for simulating stochastic chemical kinetics. , 2004, The Journal of chemical physics.

[232]  J. A. M. Janssen,et al.  The elimination of fast variables in complex chemical reactions. II. Mesoscopic level (reducible case) , 1989 .

[233]  Marc Benayoun,et al.  Avalanches in a Stochastic Model of Spiking Neurons , 2010, PLoS Comput. Biol..

[234]  Tatiana T Marquez-Lago,et al.  Generalized binomial tau-leap method for biochemical kinetics incorporating both delay and intrinsic noise. , 2008, The Journal of chemical physics.

[235]  Kevin Burrage,et al.  Numerical solutions of stochastic differential equations – implementation and stability issues , 2000 .

[236]  G. Sanguinetti,et al.  Cox process representation and inference for stochastic reaction–diffusion processes , 2016, Nature Communications.

[237]  Kevin Burrage,et al.  Stochastic simulation in systems biology , 2014, Computational and structural biotechnology journal.

[238]  Ian J. Laurenzia An analytical solution of the stochastic master equation for reversible bimolecular reaction kinetics , 2000 .

[239]  J. King,et al.  Multiscale stochastic modelling of gene expression , 2012, Journal of mathematical biology.

[240]  Wilhelm Huisinga,et al.  Hybrid Stochastic-Deterministic Solution of the Chemical Master Equation , 2012, Multiscale Model. Simul..