A reaction network scheme which implements the EM algorithm

A detailed algorithmic explanation is required for how a network of chemical reactions can generate the sophisticated behavior displayed by living cells. Though several previous works have shown that reaction networks are computationally universal and can in principle implement any algorithm, there is scope for constructions that map well onto biological reality, make efficient use of the computational potential of the native dynamics of reaction networks, and make contact with statistical mechanics. We describe a new reaction network scheme for solving a large class of statistical problems including the problem of how a cell would infer its environment from receptor-ligand bindings. Specifically we show how reaction networks can implement information projection, and consequently a generalized Expectation-Maximization algorithm, to solve maximum likelihood estimation problems in partially-observed exponential families on categorical data. Our scheme can be thought of as an algorithmic interpretation of E. T. Jaynes's vision of statistical mechanics as statistical inference.

[1]  R. Sarpeshkar.  Analog synthetic biology , 2014, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[2]  F. Horn.  Necessary and sufficient conditions for complex balancing in chemical kinetics , 1972 .

[3]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[4]  Shun-ichi Amari,et al.  Information geometry of turbo and low-density parity-check codes , 2004, IEEE Transactions on Information Theory.

[5]  Imre Csiszár,et al.  Information Theory and Statistics: A Tutorial , 2004, Found. Trends Commun. Inf. Theory.

[6]  Ryan P. Adams,et al.  Message Passing Inference with Chemical Reaction Networks , 2013, NIPS.

[7]  N. Kampen,et al.  Stochastic processes in physics and chemistry , 1981 .

[8]  Bao Quoc Tang,et al.  Trend to Equilibrium for Reaction-Diffusion Systems Arising from Complex Balanced Chemical Reaction Networks , 2016, SIAM J. Math. Anal..

[9]  L. Pachter,et al.  Algebraic Statistics for Computational Biology: Preface , 2005 .

[10]  N. Čencov.  Statistical Decision Rules and Optimal Inference , 2000 .

[11]  Eduardo Sontag,et al.  A Petri net approach to the study of persistence in chemical reaction networks. , 2006, Mathematical biosciences.

[12]  A Hjelmfelt,et al.  Chemical implementation of neural networks and Turing machines. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[13]  David F. Anderson,et al.  Product-Form Stationary Distributions for Deficiency Zero Chemical Reaction Networks , 2008, Bulletin of mathematical biology.

[14]  D. Sherrington.  Stochastic Processes in Physics and Chemistry , 1983 .

[15]  Shun-ichi Amari,et al.  Information Geometry and Its Applications , 2016 .

[16]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[17]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[18]  Ezra Miller,et al.  A Geometric Approach to the Global Attractor Conjecture , 2013, SIAM J. Appl. Dyn. Syst..

[19]  Manoj Gopalkrishnan,et al.  A Scheme for Molecular Computation of Maximum Likelihood Estimators for Log-Linear Models , 2015, DNA.

[20]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[21]  John C. Baez,et al.  Algorithmic thermodynamics , 2010, Mathematical Structures in Computer Science.

[22]  A. Zellner.  Optimal Information Processing and Bayes's Theorem , 1988 .

[23]  Manoj Gopalkrishnan,et al.  A Stochastic Molecular Scheme for an Artificial Cell to Infer Its Environment from Partial Observations , 2017, DNA.

[24]  Imre Csiszár,et al.  Information projections revisited , 2000, IEEE Trans. Inf. Theory.

[25]  K Oishi,et al.  Biomolecular implementation of linear I/O systems. , 2011, IET systems biology.

[26]  Manoj Gopalkrishnan.  Catalysis in Reaction Networks , 2011, Bulletin of mathematical biology.

[27]  Eduardo Sontag,et al.  A Petri Net Approach to Persistence Analysis in Chemical Reaction Networks , 2007 .

[28]  Norbert Wiener,et al.  Cybernetics: Control and Communication in the Animal and the Machine. , 1949 .

[29]  M. W. Birch.  Maximum Likelihood in Three-Way Contingency Tables , 1963 .

[30]  Lulu Qian,et al.  Supporting Online Material Materials and Methods Figs. S1 to S6 Tables S1 to S4 References and Notes Scaling up Digital Circuit Computation with Dna Strand Displacement Cascades , 2022 .

[31]  Martin Feinberg,et al.  On chemical kinetics of a certain class , 1972 .

[32]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[33]  Luca Cardelli,et al.  Chemical reaction network designs for asynchronous logic circuits , 2016, Natural Computing.

[34]  Georg Seelig,et al.  Molecular circuits for dynamic noise filtering , 2016, Proceedings of the National Academy of Sciences.

[35]  Matthew Cook,et al.  Computation with finite stochastic chemical reaction networks , 2008, Natural Computing.

[36]  Nick S. Jones,et al.  Chemical Boltzmann Machines , 2017, DNA.

[37]  Jehoshua Bruck,et al.  Neural network computation with DNA strand displacement cascades , 2011, Nature.

[38]  M. Tribus,et al.  Energy and information , 1971 .

[39]  Erling B. Andersen,et al.  Sufficiency and Exponential Families for Discrete Sample Spaces , 1970 .

[40]  Shun-ichi Amari,et al.  Stochastic Reasoning, Free Energy, and Information Geometry , 2004, Neural Computation.

[41]  Ho-Lin Chen,et al.  Deterministic function computation with chemical reaction networks , 2012, Natural Computing.

[42]  E. Robinson.  Cybernetics, or Control and Communication in the Animal and the Machine , 1963 .

[43]  Peter A. J. Hilbers,et al.  Computing Algebraic Functions with Biochemical Reaction Networks , 2009, Artificial Life.

[44]  Ezra Miller.  Theory and applications of lattice point methods for binomial ideals , 2010, ArXiv.