Approximation and inference methods for stochastic biochemical kinetics—a tutorial review

Stochastic fluctuations of molecule numbers are ubiquitous in biological systems. Important examples include gene expression and enzymatic processes in living cells. Such systems are typically modelled as chemical reaction networks whose dynamics are governed by the Chemical Master Equation. Despite its simple structure, no analytic solutions to the Chemical Master Equation are known for most systems. Moreover, stochastic simulations are computationally expensive, making systematic analysis and statistical inference a challenging task. Consequently, significant effort has been spent in recent decades on the development of efficient approximation and inference methods. This article gives an introduction to basic modelling concepts as well as an overview of state of the art methods. First, we motivate and introduce deterministic and stochastic methods for modelling chemical networks, and give an overview of simulation and exact solution methods. Next, we discuss several approximation methods, including the chemical Langevin equation, the system size expansion, moment closure approximations, time-scale separation approximations and hybrid methods. We discuss their various properties and review recent advances and remaining challenges for these methods. We present a comparison of several of these methods by means of a numerical case study and highlight some of their respective advantages and disadvantages. Finally, we discuss the problem of inference from experimental data in the Bayesian framework and review recent methods developed the literature. In summary, this review gives a self-contained introduction to modelling, approximations and inference methods for stochastic chemical kinetics.

[1]  Andreas Hellander,et al.  Hybrid method for the chemical master equation , 2007, J. Comput. Phys..

[2]  J. Tóth,et al.  A full stochastic description of the Michaelis-Menten reaction for small systems. , 1977, Acta biochimica et biophysica; Academiae Scientiarum Hungaricae.

[3]  C. Rao,et al.  Stochastic chemical kinetics and the quasi-steady-state assumption: Application to the Gillespie algorithm , 2003 .

[4]  S. Isaacson Relationship between the reaction–diffusion master equation and particle tracking models , 2008 .

[5]  Paul D. W. Kirk,et al.  MEANS: python package for Moment Expansion Approximation, iNference and Simulation , 2016, Bioinform..

[6]  L. Petzold,et al.  Reaction-diffusion master equation in the microscopic limit. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  Leo A. Goodman,et al.  Population Growth of the Sexes , 1953 .

[8]  Andrew Golightly,et al.  Bayesian inference for hybrid discrete-continuous stochastic kinetic models , 2014, 1402.6602.

[9]  P. Swain,et al.  Gene Regulation at the Single-Cell Level , 2005, Science.

[10]  Dan ie l T. Gil lespie A rigorous derivation of the chemical master equation , 1992 .

[11]  Holger Fröhlich,et al.  Modeling ERBB receptor-regulated G1/S transition to find novel targets for de novo trastuzumab resistance , 2009, BMC Systems Biology.

[12]  Darren J. Wilkinson,et al.  Bayesian inference for a discretely observed stochastic kinetic model , 2008, Stat. Comput..

[13]  Ruth J. Williams,et al.  Correlation resonance generated by coupled enzymatic processing. , 2010, Biophysical journal.

[14]  M. Elowitz,et al.  Functional roles for noise in genetic circuits , 2010, Nature.

[15]  J. A. M. Janssen,et al.  The elimination of fast variables in complex chemical reactions. III. Mesoscopic level (irreducible case) , 1989 .

[16]  Michael P H Stumpf,et al.  A general moment expansion method for stochastic kinetic models. , 2013, The Journal of chemical physics.

[17]  A. J. McKane,et al.  Stochastic models of evolution in genetics, ecology and linguistics , 2007, cond-mat/0703478.

[18]  Ohira,et al.  Master-equation approach to stochastic neurodynamics. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[19]  Sean R. Anderson,et al.  Repelled from the wound, or randomly dispersed? Reverse migration behaviour of neutrophils characterized by dynamic modelling , 2012, Journal of The Royal Society Interface.

[20]  Mudita Singhal,et al.  COPASI - a COmplex PAthway SImulator , 2006, Bioinform..

[21]  Ian J. Laurenzia An analytical solution of the stochastic master equation for reversible bimolecular reaction kinetics , 2000 .

[22]  Jae Kyoung Kim,et al.  The relationship between stochastic and deterministic quasi-steady state approximations , 2015, BMC Systems Biology.

[23]  Marco Beccuti,et al.  Analysis of Petri Net Models through Stochastic Differential Equations , 2014, Petri Nets.

[24]  Niraj Kumar,et al.  Exact distributions for stochastic gene expression models with bursting and feedback. , 2014, Physical review letters.

[25]  João Pedro Hespanha,et al.  Approximate Moment Dynamics for Chemically Reacting Systems , 2011, IEEE Transactions on Automatic Control.

[26]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[27]  G. Sanguinetti,et al.  Cox process representation and inference for stochastic reaction–diffusion processes , 2016, Nature Communications.

[28]  Tobias Jahnke,et al.  On Reduced Models for the Chemical Master Equation , 2011, Multiscale Model. Simul..

[29]  P. Swain,et al.  Intrinsic and extrinsic contributions to stochasticity in gene expression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[30]  D. Vlachos,et al.  Binomial distribution based tau-leap accelerated stochastic simulation. , 2005, The Journal of chemical physics.

[31]  M. Scott,et al.  Approximating intrinsic noise in continuous multispecies models , 2011, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[32]  Todd K. Leen,et al.  Stochastic Perturbation Methods for Spike-Timing-Dependent Plasticity , 2012, Neural Computation.

[33]  Marco Beccuti,et al.  Approximate analysis of biological systems by hybrid switching jump diffusion , 2014, Theor. Comput. Sci..

[34]  D. Sherrington Stochastic Processes in Physics and Chemistry , 1983 .

[35]  Philipp Thomas,et al.  Stochastic Simulation of Biomolecular Networks in Dynamic Environments , 2015, PLoS Comput. Biol..

[36]  Mark A. Stalzer,et al.  Efficient Formulations for Exact Stochastic Simulation of Chemical Systems , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[37]  D. Talay Numerical solution of stochastic differential equations , 1994 .

[38]  David F. Anderson,et al.  Product-Form Stationary Distributions for Deficiency Zero Chemical Reaction Networks , 2008, Bulletin of mathematical biology.

[39]  P. R. ten Wolde,et al.  Simulating biochemical networks at the particle level and in time and space: Green's function reaction dynamics. , 2005, Physical review letters.

[40]  R. Grima,et al.  How reliable is the linear noise approximation of gene regulatory networks? , 2013, BMC Genomics.

[41]  M J Keeling,et al.  Multiplicative moments and measures of persistence in ecology. , 2000, Journal of theoretical biology.

[42]  Alain Destexhe,et al.  A Master Equation Formalism for Macroscopic Modeling of Asynchronous Irregular Activity States , 2009, Neural Computation.

[43]  Hamid Bolouri,et al.  Dizzy: Stochastic Simulation of Large-scale Genetic Regulatory Networks , 2005, J. Bioinform. Comput. Biol..

[44]  M. Keeling,et al.  On methods for studying stochastic disease dynamics , 2008, Journal of The Royal Society Interface.

[45]  Philipp Thomas,et al.  Computation of biochemical pathway fluctuations beyond the linear noise approximation using iNA , 2012, 2012 IEEE International Conference on Bioinformatics and Biomedicine.

[46]  Guido Sanguinetti,et al.  Unbiased Bayesian inference for population Markov jump processes via random truncations , 2015, Stat. Comput..

[47]  J.P. Hespanha,et al.  Lognormal Moment Closures for Biochemical Reactions , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[48]  R. F. Pawula,et al.  Approximation of the Linear Boltzmann Equation by the Fokker-Planck Equation , 1967 .

[49]  H. Wu,et al.  Thermal transport through a one-dimensional quantum spin-1/2 chain heterostructure: The role of three-site spin interaction , 2012 .

[50]  R. Grima,et al.  An effective rate equation approach to reaction kinetics in small volumes: theory and application to biochemical reactions in nonequilibrium steady-state conditions. , 2010, The Journal of chemical physics.

[51]  J Halloy,et al.  Deterministic Versus Stochastic Models for Circadian Rhythms , 2002, Journal of biological physics.

[52]  K. Ishida,et al.  Stochastic Model for Bimolecular Reaction , 1964 .

[53]  Chetan D. Pahlajani,et al.  Stochastic reduction method for biological chemical kinetics using time-scale separation. , 2011, Journal of theoretical biology.

[54]  Jürgen Pahle,et al.  Biochemical simulations: stochastic, approximate stochastic and hybrid approaches , 2008, Briefings Bioinform..

[55]  Wolfgang Weidlich,et al.  Sociodynamics: a Systematic Approach to Mathematical Modelling in the Social Sciences , 2000 .

[56]  J. Goutsias,et al.  Numerical Integration of the Master Equation in Some Models of Stochastic Epidemiology , 2012, PloS one.

[57]  J. A. M. Janssen,et al.  The elimination of fast variables in complex chemical reactions. II. Mesoscopic level (reducible case) , 1989 .

[58]  L. A. Segel,et al.  The Quasi-Steady-State Assumption: A Case Study in Perturbation , 1989, SIAM Rev..

[59]  Eric Bonabeau,et al.  Agent-based modeling: Methods and techniques for simulating human systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[60]  W. Ebeling Stochastic Processes in Physics and Chemistry , 1995 .

[61]  Vahid Shahrezaei,et al.  Analytical distributions for stochastic gene expression , 2008, Proceedings of the National Academy of Sciences.

[62]  S. Jonathan Chapman,et al.  Analysis of Brownian Dynamics Simulations of Reversible Bimolecular Reactions , 2010, SIAM J. Appl. Math..

[63]  Guido Sanguinetti,et al.  Approximate inference in latent Gaussian-Markov models from continuous time observations , 2013, NIPS.

[64]  Abhyudai Singh,et al.  Conditional Moment Closure Schemes for Studying Stochastic Dynamics of Genetic Circuits , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[65]  Philipp Thomas,et al.  Rigorous elimination of fast stochastic variables from the linear noise approximation using projection operators. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[66]  Ramon Grima,et al.  Analytical approximations for spatial stochastic gene expression in single cells and tissues , 2016, Journal of The Royal Society Interface.

[67]  Peter Whittle,et al.  Systems in stochastic equilibrium , 1986 .

[68]  Pedro Mendes,et al.  Biochemical fluctuations, optimisation and the linear noise approximation , 2012, BMC Systems Biology.

[69]  J. Goutsias,et al.  Markovian dynamics on complex reaction networks , 2012, 1205.5524.

[70]  Guido Sanguinetti,et al.  Learning and Designing Stochastic Processes from Logical Constraints , 2015, Log. Methods Comput. Sci..

[71]  Christian P. Robert,et al.  Statistics for Spatio-Temporal Data , 2014 .

[72]  Cosmin Safta,et al.  Hybrid discrete/continuum algorithms for stochastic reaction networks , 2015, J. Comput. Phys..

[73]  Hong Qian,et al.  Grand canonical Markov model: a stochastic theory for open nonequilibrium biochemical networks. , 2006, The Journal of chemical physics.

[74]  Tobias Jahnke,et al.  Error Bound for Piecewise Deterministic Processes Modeling Stochastic Reaction Systems , 2012, Multiscale Model. Simul..

[75]  K. Zygalakis,et al.  Fast stochastic simulation of biochemical reaction systems by alternative formulations of the chemical Langevin equation. , 2010, The Journal of chemical physics.

[76]  Darren J. Wilkinson,et al.  Moment closure based parameter inference of stochastic kinetic models , 2013, Stat. Comput..

[77]  A. McKane,et al.  Amplified Biochemical Oscillations in Cellular Systems , 2006, q-bio/0604001.

[78]  Louise Dyson,et al.  Noise-induced bistable states and their mean switching time in foraging colonies. , 2013, Physical review letters.

[79]  Donald A. McQuarrie,et al.  Kinetics of Small Systems. I , 1963 .

[80]  Muruhan Rathinam,et al.  Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method , 2003 .

[81]  P. Staff,et al.  Stochastic Approach to First-Order Chemical Reaction Kinetics , 1966 .

[82]  Verena Wolf,et al.  Model Reconstruction for Moment-Based Stochastic Chemical Kinetics , 2015, ACM Trans. Model. Comput. Simul..

[83]  P. Kloeden,et al.  Numerical Solution of Stochastic Differential Equations , 1992 .

[84]  Ovidiu Radulescu,et al.  Hybrid stochastic simplifications for multiscale gene networks , 2009, BMC Systems Biology.

[85]  Deena R. Schmidt,et al.  Steady-state fluctuations of a genetic feedback loop: an exact solution. , 2012, The Journal of chemical physics.

[86]  David F Anderson,et al.  A modified next reaction method for simulating chemical systems with time dependent propensities and delays. , 2007, The Journal of chemical physics.

[87]  J. E. Moyal Stochastic Processes and Statistical Physics , 1949 .

[88]  Roland Eils,et al.  General Stochastic Hybrid Method for the Simulation of Chemical Reaction Processes in Cells , 2004, CMSB.

[89]  Wilhelm Huisinga,et al.  ADAPTIVE SIMULATION OF HYBRID STOCHASTIC AND DETERMINISTIC MODELS FOR BIOCHEMICAL SYSTEMS , 2005 .

[90]  Radek Erban,et al.  Error Analysis of Diffusion Approximation Methods for Multiscale Systems in Reaction Kinetics , 2014, SIAM J. Sci. Comput..

[91]  Radek Erban,et al.  Tensor methods for parameter estimation and bifurcation analysis of stochastic reaction networks , 2015, Journal of The Royal Society Interface.

[92]  Yee Whye Teh,et al.  Fast MCMC sampling for Markov jump processes and extensions , 2012, J. Mach. Learn. Res..

[93]  Guido Sanguinetti,et al.  Reconstructing transcription factor activities in hierarchical transcription network motifs , 2011, Bioinform..

[94]  J. Lygeros,et al.  Moment-based inference predicts bimodality in transient gene expression , 2012, Proceedings of the National Academy of Sciences.

[95]  Sheng Wu,et al.  StochKit2: software for discrete stochastic simulation of biochemical systems with events , 2011, Bioinform..

[96]  C. Bianca,et al.  Evaluation of reaction fluxes in stationary and oscillating far-from-equilibrium biological systems , 2015 .

[97]  Jeffrey W. Smith,et al.  Stochastic Gene Expression in a Single Cell , .

[98]  Linda R Petzold,et al.  Adaptive explicit-implicit tau-leaping method with automatic tau selection. , 2007, The Journal of chemical physics.

[99]  Paulette Clancy,et al.  A "partitioned leaping" approach for multiscale modeling of chemical reaction dynamics. , 2006, The Journal of chemical physics.

[100]  Radek Erban,et al.  Hybrid framework for the simulation of stochastic chemical kinetics , 2016, J. Comput. Phys..

[101]  Ertugrul M. Ozbudak,et al.  Regulation of noise in the expression of a single gene , 2002, Nature Genetics.

[102]  P. Swain,et al.  Stochastic Gene Expression in a Single Cell , 2002, Science.

[103]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[104]  Giancarlo Mauri,et al.  The Interplay of Intrinsic and Extrinsic Bounded Noises in Biomolecular Networks , 2012, PloS one.

[105]  Guido Sanguinetti,et al.  Hybrid regulatory models: a statistically tractable approach to model regulatory network dynamics , 2013, Bioinform..

[106]  D G Vlachos,et al.  Overcoming stiffness in stochastic simulation stemming from partial equilibrium: a multiscale Monte Carlo algorithm. , 2005, The Journal of chemical physics.

[107]  J. Bowen,et al.  Singular perturbation refinement to quasi-steady state approximation in chemical kinetics , 1963 .

[108]  D. Fanelli,et al.  Analytical study of non Gaussian fluctuations in a stochastic scheme of autocatalytic reactions , 2011, 1104.5668.

[109]  J. Timmer,et al.  Noisy signaling through promoter logic gates. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[110]  R. Jackson,et al.  General mass action kinetics , 1972 .

[111]  J. Hillston,et al.  Stochastic properties of the plant circadian clock , 2012, Journal of The Royal Society Interface.

[112]  R. Law,et al.  A Jump-Growth Model for Predator–Prey Dynamics: Derivation and Application to Marine Ecosystems , 2008, Bulletin of mathematical biology.

[113]  Sheng Wu,et al.  The time dependent propensity function for acceleration of spatial stochastic simulation of reaction-diffusion systems , 2014, J. Comput. Phys..

[114]  Duccio Fanelli,et al.  Enhanced stochastic oscillations in autocatalytic reactions. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[115]  Roger B Sidje,et al.  Understanding the finite state projection and related methods for solving the chemical master equation , 2016, Physical biology.

[116]  Hong Li,et al.  Algorithms and Software for Stochastic Simulation of Biochemical Reacting Systems , 2008, Biotechnology progress.

[117]  M. S. Bartlett,et al.  Some Evolutionary Stochastic Processes , 1949 .

[118]  Yiannis Kaznessis,et al.  Accurate hybrid stochastic simulation of a system of coupled chemical or biochemical reactions. , 2005, The Journal of chemical physics.

[119]  M. Doi Stochastic theory of diffusion-controlled reaction , 1976 .

[120]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[121]  Guido Sanguinetti,et al.  Comparison of different moment-closure approximations for stochastic chemical kinetics. , 2015, The Journal of chemical physics.

[122]  M. Feinberg The existence and uniqueness of steady states for a class of chemical reaction networks , 1995 .

[123]  Joseph D Challenger,et al.  Synchronization of stochastic oscillators in biochemical systems. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[124]  Guido Sanguinetti,et al.  The complex chemical Langevin equation. , 2014, The Journal of chemical physics.

[125]  Michael P H Stumpf,et al.  Multivariate moment closure techniques for stochastic kinetic models. , 2015, The Journal of chemical physics.

[126]  M. Delbrück Statistical Fluctuations in Autocatalytic Reactions , 1940 .

[127]  P. Whittle On the Use of the Normal Approximation in the Treatment of Stochastic Processes , 1957 .

[128]  C Jayaprakash,et al.  Mixed Poisson distributions in exact solutions of stochastic autoregulation models. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[129]  Philipp Thomas,et al.  Communication: limitations of the stochastic quasi-steady-state approximation in open biochemical reaction networks. , 2011, The Journal of chemical physics.

[130]  C. Gillespie Moment-closure approximations for mass-action models. , 2009, IET systems biology.

[131]  T. Kurtz Limit theorems and diffusion approximations for density dependent Markov chains , 1976 .

[132]  Ian Stark,et al.  The Continuous pi-Calculus: A Process Algebra for Biochemical Modelling , 2008, CMSB.

[133]  E. O’Shea,et al.  Living with noisy genes: how cells function reliably with inherent variability in gene expression. , 2007, Annual review of biophysics and biomolecular structure.

[134]  Chin-Kun Hu,et al.  Nonequilibrium Lyapunov function and a fluctuation relation for stochastic systems: Poisson-representation approach. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[135]  Byron F. Brehm-Stecher,et al.  Single-Cell Microbiology: Tools, Technologies, and Applications , 2004, Microbiology and Molecular Biology Reviews.

[136]  G. Verghese,et al.  Enhanced identification and exploitation of time scales for model reduction in stochastic chemical kinetics. , 2008, The Journal of chemical physics.

[137]  R. Heinrich,et al.  The Regulation of Cellular Systems , 1996, Springer US.

[138]  A. Kierzek,et al.  Bridging the gap between stochastic and deterministic regimes in the kinetic simulations of the biochemical reaction networks. , 2004, Biophysical journal.

[139]  David A. Rand,et al.  A stochastic transcriptional switch model for single cell imaging data , 2015, Biostatistics.

[140]  H. Kramers Brownian motion in a field of force and the diffusion model of chemical reactions , 1940 .

[141]  Philipp Thomas,et al.  The slow-scale linear noise approximation: an accurate, reduced stochastic description of biochemical networks under timescale separation conditions , 2012, BMC Systems Biology.

[142]  Aleksandar Donev,et al.  A First-Passage Kinetic Monte Carlo algorithm for complex diffusion-reaction systems , 2009, J. Comput. Phys..

[143]  Kevin Burrage,et al.  Stochastic simulation in systems biology , 2014, Computational and structural biotechnology journal.

[144]  Axel Kowald,et al.  Systems Biology in Practice: Concepts, Implementation and Application , 2005 .

[145]  J. Rawlings,et al.  Approximate simulation of coupled fast and slow reactions for stochastic chemical kinetics , 2002 .

[146]  P. T. Wolde,et al.  Simulating biochemical networks at the particle level and in time and space: Green's function reaction dynamics. , 2005 .

[147]  M. L. Simpson,et al.  Gene network shaping of inherent noise spectra , 2006, Nature.

[148]  Philipp Thomas,et al.  Approximate probability distributions of the master equation. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[149]  M. Doi Second quantization representation for classical many-particle system , 1976 .

[150]  Corrections to the law of mass action and properties of the asymptotic t = ' state for reversible diffusion-limited reactions , 2005, cond-mat/0503198.

[151]  Linda R Petzold,et al.  The slow-scale stochastic simulation algorithm. , 2005, The Journal of chemical physics.

[152]  Natasha A. Neogi,et al.  Dynamic Partitioning of Large Discrete Event Biological Systems for Hybrid Simulation and Analysis , 2004, HSCC.

[153]  Erwin Frey,et al.  Master equations and the theory of stochastic path integrals , 2016, Reports on progress in physics. Physical Society.

[154]  Guido Sanguinetti,et al.  Validity conditions for moment closure approximations in stochastic chemical kinetics. , 2014, The Journal of chemical physics.

[155]  C. Pesce,et al.  Regulated cell-to-cell variation in a cell-fate decision system , 2005, Nature.

[156]  Ilya Nemenman,et al.  Adiabatic coarse-graining and simulations of stochastic biochemical networks , 2009, Proceedings of the National Academy of Sciences.

[157]  Kevin Burrage,et al.  Stochastic approaches for modelling in vivo reactions , 2004, Comput. Biol. Chem..

[158]  Tatiana T Marquez-Lago,et al.  Generalized binomial tau-leap method for biochemical kinetics incorporating both delay and intrinsic noise. , 2008, The Journal of chemical physics.

[159]  Desmond J. Higham,et al.  An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations , 2001, SIAM Rev..

[160]  K. Burrage,et al.  Numerical methods for strong solutions of stochastic differential equations: an overview , 2004, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[161]  Andreas Ruttor,et al.  Approximate parameter inference in a stochastic reaction-diffusion model , 2010, AISTATS.

[162]  S. Frühwirth-Schnatter Data Augmentation and Dynamic Linear Models , 1994 .

[163]  Anthony F. Bartholomay,et al.  Stochastic models for chemical reactions: I. Theory of the unimolecular reaction process , 1958 .

[164]  Thomas A. Henzinger,et al.  Hybrid Systems: Computation and Control , 1998, Lecture Notes in Computer Science.

[165]  Philipp Thomas,et al.  Distribution Approximations for the Chemical Master Equation: Comparison of the Method of Moments and the System Size Expansion , 2015, 1509.09104.

[166]  N. Kampen,et al.  Stochastic processes in physics and chemistry , 1981 .

[167]  Ramon Grima,et al.  A study of the accuracy of moment-closure approximations for stochastic chemical kinetics. , 2012, The Journal of chemical physics.

[168]  Mark A. Girolami,et al.  Markov chain Monte Carlo inference for Markov jump processes via the linear noise approximation , 2012, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[169]  D. Gillespie The chemical Langevin equation , 2000 .

[170]  Guido Sanguinetti,et al.  Expectation propagation for continuous time stochastic processes , 2015, 1512.06098.

[171]  S. Isaacson A convergent reaction-diffusion master equation. , 2012, Journal of Chemical Physics.

[172]  Van Kampen,et al.  The Expansion of the Master Equation , 2007 .

[173]  Wilhelm Huisinga,et al.  Hybrid Stochastic-Deterministic Solution of the Chemical Master Equation , 2012, Multiscale Model. Simul..

[174]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[175]  Guido Sanguinetti,et al.  Variational inference for Markov jump processes , 2007, NIPS.

[176]  Erika Cule,et al.  ABC-SysBio—approximate Bayesian computation in Python with GPU support , 2010, Bioinform..

[177]  Darren J. Wilkinson,et al.  CaliBayes and BASIS: integrated tools for the calibration, simulation and storage of biological simulation models , 2010, Briefings Bioinform..

[178]  G. Briggs,et al.  A Note on the Kinetics of Enzyme Action. , 1925, The Biochemical journal.

[179]  P. Waage,et al.  Studies concerning affinity , 1986 .

[180]  Visakan Kadirkamanathan,et al.  Parameter estimation and inference for stochastic reaction-diffusion systems: application to morphogenesis in D. melanogaster , 2009, BMC Systems Biology.

[181]  A. V. Grimstone Molecular biology of the cell (3rd edn) , 1995 .

[182]  G. Verghese,et al.  Mass fluctuation kinetics: capturing stochastic effects in systems of chemical reactions through coupled mean-variance computations. , 2007, The Journal of chemical physics.

[183]  A. Duncan,et al.  Noise-induced multistability in chemical systems: Discrete versus continuum modeling. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[184]  Julien F. Ollivier,et al.  Colored extrinsic fluctuations and stochastic gene expression , 2008, Molecular systems biology.

[185]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[186]  Andreas Hellander,et al.  Perspective: Stochastic algorithms for chemical kinetics. , 2013, The Journal of chemical physics.

[187]  D. Wilkinson,et al.  Bayesian Inference for Stochastic Kinetic Models Using a Diffusion Approximation , 2005, Biometrics.

[188]  N. Popović,et al.  Phenotypic switching in gene regulatory networks , 2014, Proceedings of the National Academy of Sciences.

[189]  Peter G. Hufton,et al.  Intrinsic noise in systems with switching environments. , 2015, Physical review. E.

[190]  Y. Wong,et al.  Positivity preserving chemical Langevin equations , 2008 .

[191]  Ramon Grima,et al.  Model reduction for stochastic chemical systems with abundant species. , 2015, The Journal of chemical physics.

[192]  M. Khammash,et al.  The finite state projection algorithm for the solution of the chemical master equation. , 2006, The Journal of chemical physics.

[193]  K. Burrage,et al.  Binomial leap methods for simulating stochastic chemical kinetics. , 2004, The Journal of chemical physics.

[194]  Masaru Tomita,et al.  A multi-algorithm, multi-timescale method for cell simulation , 2004, Bioinform..

[195]  Andreas Ruttor,et al.  Inference in continuous-time change-point models , 2011, NIPS.

[196]  LUKASZ SZPRUCH,et al.  Comparing Hitting Time Behavior of Markov Jump Processes and Their Diffusion Approximations , 2010, Multiscale Model. Simul..

[197]  김삼묘,et al.  “Bioinformatics” 특집을 내면서 , 2000 .

[198]  J. Timmer,et al.  Signatures of nonlinearity in single cell noise-induced oscillations. , 2013, Journal of theoretical biology.

[199]  Eric Vanden-Eijnden,et al.  Nested stochastic simulation algorithm for chemical kinetic systems with disparate rates. , 2005, The Journal of chemical physics.

[200]  Paul J. Choi,et al.  Quantifying E. coli Proteome and Transcriptome with Single-Molecule Sensitivity in Single Cells , 2010, Science.

[201]  P R Taylor,et al.  Deriving appropriate boundary conditions, and accelerating position-jump simulations, of diffusion using non-local jumping , 2014, Physical biology.

[202]  D. Gillespie,et al.  A diffusional bimolecular propensity function. , 2009, The Journal of chemical physics.

[203]  N. Kampen The equilibrium distribution of a chemical mixture , 1976 .

[204]  P. Waage,et al.  Ueber die chemische Affinität. § 1. Einleitung , 1879 .

[205]  C. Gardiner Handbook of Stochastic Methods , 1983 .

[206]  Yiannis N Kaznessis,et al.  An equation-free probabilistic steady-state approximation: dynamic application to the stochastic simulation of biochemical reaction networks. , 2005, The Journal of chemical physics.

[207]  Malbor Asllani,et al.  The linear noise approximation for reaction-diffusion systems on networks , 2013, 1305.7318.

[208]  R. Erban,et al.  Stochastic modelling of reaction–diffusion processes: algorithms for bimolecular reactions , 2009, Physical biology.

[209]  Paul Fearnhead,et al.  Inference for reaction networks using the linear noise approximation , 2012, Biometrics.

[210]  Javad Usefie Mafahim,et al.  Stochastic Wilson – Cowan models of neuronal network dynamics with memory and delay , 2015 .

[211]  Yiannis N. Kaznessis,et al.  A closure scheme for chemical master equations , 2013, Proceedings of the National Academy of Sciences.

[212]  Ian J. Laurenzi,et al.  An analytical solution of the stochastic master equation for reversible bimolecular reaction kinetics , 2000 .

[213]  D. Gillespie A rigorous derivation of the chemical master equation , 1992 .

[214]  D. Gillespie A General Method for Numerically Simulating the Stochastic Time Evolution of Coupled Chemical Reactions , 1976 .

[215]  Kevin Burrage,et al.  Numerical solutions of stochastic differential equations – implementation and stability issues , 2000 .

[216]  Guido Sanguinetti,et al.  Efficient Stochastic Simulation of Systems with Multiple Time Scales via Statistical Abstraction , 2015, CMSB.

[217]  Fabian J Theis,et al.  Method of conditional moments (MCM) for the Chemical Master Equation , 2013, Journal of Mathematical Biology.

[218]  Guido Sanguinetti,et al.  Learning and Designing Stochastic Processes from Logical Constraints , 2013, QEST.

[219]  M. Scott,et al.  Non-linear corrections to the time-covariance function derived from a multi-state chemical master equation. , 2012, IET systems biology.

[220]  Carlos González-Alcón,et al.  Modeling of leishmaniasis infection dynamics: novel application to the design of effective therapies , 2012, BMC Systems Biology.

[221]  Cleve B. Moler,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later , 1978, SIAM Rev..

[222]  J. Elf,et al.  Lost in presumption: stochastic reactions in spatial models , 2012, Nature Methods.

[223]  T. Kurtz,et al.  Separation of time-scales and model reduction for stochastic reaction networks. , 2010, 1011.1672.

[224]  D. Gillespie Exact Stochastic Simulation of Coupled Chemical Reactions , 1977 .

[225]  Ioannis G Kevrekidis,et al.  A constrained approach to multiscale stochastic simulation of chemically reacting systems. , 2011, The Journal of chemical physics.

[226]  E. O’Shea,et al.  Noise in protein expression scales with natural protein abundance , 2006, Nature Genetics.

[227]  C. Gardiner,et al.  The poisson representation. I. A new technique for chemical master equations , 1977 .

[228]  Michael A. Buice,et al.  Systematic Fluctuation Expansion for Neural Network Activity Equations , 2009, Neural Computation.

[229]  Fabian J. Theis,et al.  CERENA: ChEmical REaction Network Analyzer—A Toolbox for the Simulation and Analysis of Stochastic Chemical Kinetics , 2016, PloS one.

[230]  R. Grima,et al.  Intrinsic Noise Analyzer: A Software Package for the Exploration of Stochastic Biochemical Kinetics Using the System Size Expansion , 2012, PloS one.

[231]  Corrado Priami,et al.  Snazer: the simulations and networks analyzer , 2010, BMC Systems Biology.

[232]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[233]  István Simon,et al.  Self-regulating genes. Exact steady state solution by using Poisson representation , 2013, 1312.3919.

[234]  João P. Hespanha,et al.  Equilibrium distributions of simple biochemical reaction systems for time-scale separation in stochastic reaction networks , 2014, Journal of The Royal Society Interface.

[235]  Piet Van Mieghem,et al.  Epidemic processes in complex networks , 2014, ArXiv.

[236]  Darren J Wilkinson,et al.  Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo , 2011, Interface Focus.

[237]  Kevin R. Sanft,et al.  Legitimacy of the stochastic Michaelis-Menten approximation. , 2011, IET systems biology.

[238]  Angelo Vulpiani,et al.  Coarse graining of master equations with fast and slow states. , 2008, The Journal of chemical physics.

[239]  W. Huisinga,et al.  Solving the chemical master equation for monomolecular reaction systems analytically , 2006, Journal of mathematical biology.

[240]  A. Oudenaarden,et al.  Cellular Decision Making and Biological Noise: From Microbes to Mammals , 2011, Cell.

[241]  P J Staff A stochastic development of the reversible Michaelis-Menten mechanism. , 1970, Journal of theoretical biology.

[242]  Daniel T. Gillespie,et al.  Simulation Methods in Systems Biology , 2008, SFM.

[243]  Todd K. Leen,et al.  Perturbation theory for stochastic learning dynamics , 2011, The 2011 International Joint Conference on Neural Networks.

[244]  N. Bailey,et al.  A simple stochastic epidemic. , 1950, Biometrika.

[245]  Heinz Koeppl,et al.  Jump-Diffusion Approximation of Stochastic Reaction Dynamics: Error Bounds and Algorithms , 2014, Multiscale Model. Simul..

[246]  Linda R. Petzold,et al.  Stochastic modelling of gene regulatory networks , 2005 .

[247]  Christian Kohlschein An introduction to Hidden Markov Models , 2007 .

[248]  Ingemar Nåsell,et al.  An extension of the moment closure method. , 2003, Theoretical population biology.

[249]  Marc Benayoun,et al.  Emergent Oscillations in Networks of Stochastic Spiking Neurons , 2011, PloS one.

[250]  J. Hespanha Moment closure for biochemical networks , 2008, 2008 3rd International Symposium on Communications, Control and Signal Processing.

[251]  M. Thattai,et al.  Intrinsic noise in gene regulatory networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[252]  Marc Benayoun,et al.  Avalanches in a Stochastic Model of Spiking Neurons , 2010, PLoS Comput. Biol..

[253]  Yi-fei Wang,et al.  Efficient binomial leap method for simulating chemical kinetics. , 2007, The Journal of chemical physics.

[254]  A. McKane,et al.  Stochastic formulation of ecological models and their applications. , 2012, Trends in ecology & evolution.

[255]  Guido Sanguinetti,et al.  Expectation propagation for diffusion processes by moment closure approximations , 2015 .

[256]  Xiaodong Cai,et al.  K-leap method for accelerating stochastic simulation of coupled chemical reactions. , 2007, The Journal of chemical physics.

[257]  J. Goutsias Quasiequilibrium approximation of fast reaction kinetics in stochastic biochemical systems. , 2005, The Journal of chemical physics.

[258]  Brian Munsky,et al.  Reduction and solution of the chemical master equation using time scale separation and finite state projection. , 2006, The Journal of chemical physics.

[259]  Haluk Resat,et al.  Multinomial tau-leaping method for stochastic kinetic simulations. , 2007, The Journal of chemical physics.

[260]  Timo R. Maarleveld,et al.  StochPy: A Comprehensive, User-Friendly Tool for Simulating Stochastic Biological Processes , 2013, PloS one.

[261]  Tianhai Tian,et al.  A multi-scaled approach for simulating chemical reaction systems. , 2004, Progress in biophysics and molecular biology.

[262]  E. Crow,et al.  Lognormal Distributions: Theory and Applications , 1987 .

[263]  Philipp Thomas,et al.  Stochastic theory of large-scale enzyme-reaction networks: finite copy number corrections to rate equation models. , 2010, The Journal of chemical physics.

[264]  Andrew J. Black,et al.  Stochasticity in staged models of epidemics: quantifying the dynamics of whooping cough , 2010, Journal of The Royal Society Interface.

[265]  Wolfgang Weidlich,et al.  Physics and social science — The approach of synergetics , 1991 .

[266]  J. King,et al.  Multiscale stochastic modelling of gene expression , 2012, Journal of mathematical biology.

[267]  Saswati Dana,et al.  Physically consistent simulation of mesoscale chemical kinetics: The non-negative FIS-α method , 2011, J. Comput. Phys..

[268]  M. Peter,et al.  Scalable inference of heterogeneous reaction kinetics from pooled single-cell recordings , 2013, Nature Methods.

[269]  Ovidiu Radulescu,et al.  Convergence of stochastic gene networks to hybrid piecewise deterministic processes , 2011, 1101.1431.

[270]  Andreas Ruttor,et al.  Efficient statistical inference for stochastic reaction processes. , 2009, Physical review letters.

[271]  Upinder S. Bhalla,et al.  Adaptive stochastic-deterministic chemical kinetic simulations , 2004, Bioinform..

[272]  Carsten Marr,et al.  A geometric analysis of fast-slow models for stochastic gene expression , 2015, Journal of Mathematical Biology.

[273]  D. Gillespie Approximate accelerated stochastic simulation of chemically reacting systems , 2001 .

[274]  Ankit Gupta,et al.  Adaptive hybrid simulations for multiscale stochastic reaction networks. , 2014, The Journal of chemical physics.

[275]  H. El-Samad,et al.  A rigorous framework for multiscale simulation of stochastic cellular networks. , 2009, The Journal of chemical physics.

[276]  E Weinan,et al.  Nested stochastic simulation algorithms for chemical kinetic systems with multiple time scales , 2007, J. Comput. Phys..

[277]  Samuel A. Isaacson,et al.  The Reaction-Diffusion Master Equation as an Asymptotic Approximation of Diffusion to a Small Target , 2009, SIAM J. Appl. Math..

[278]  Awad H. Al-Mohy,et al.  Computing the Action of the Matrix Exponential, with an Application to Exponential Integrators , 2011, SIAM J. Sci. Comput..