Intrinsic Computation of a Monod-Wyman-Changeux Molecule

Causal states are minimal sufficient statistics of prediction of a stochastic process, their coding cost is called statistical complexity, and the implied causal structure yields a sense of the process’ “intrinsic computation”. We discuss how statistical complexity changes with slight changes to the underlying model– in this case, a biologically-motivated dynamical model, that of a Monod-Wyman-Changeux molecule. Perturbations to kinetic rates cause statistical complexity to jump from finite to infinite. The same is not true for excess entropy, the mutual information between past and future, or for the molecule’s transfer function. We discuss the implications of this for the relationship between intrinsic and functional computation of biological sensory systems.

[1]  James Odell,et al.  Between order and chaos , 2011, Nature Physics.

[2]  Mark D. Plumbley,et al.  A measure of statistical complexity based on predictive information , 2010, ArXiv.

[3]  James P. Crutchfield,et al.  Computational Mechanics: Pattern and Prediction, Structure and Simplicity , 1999, ArXiv.

[4]  J. Changeux.  50 years of allosteric interactions: the twists and turns of the models , 2013, Nature Reviews Molecular Cell Biology.

[5]  James P. Crutchfield,et al.  Prediction, Retrodiction, and the Amount of Information Stored in the Present , 2009, ArXiv.

[6]  James P. Crutchfield,et al.  Informational and Causal Architecture of Discrete-Time Renewal Processes , 2014, Entropy.

[7]  Devavrat Shah,et al.  On entropy for mixtures of discrete and continuous variables , 2006, ArXiv.

[8]  James P. Crutchfield,et al.  Time resolution dependence of information measures for spiking neurons: scaling and universality , 2015, Front. Comput. Neurosci..

[9]  Michael J. Berry,et al.  Predictive information in a sensory population , 2013, Proceedings of the National Academy of Sciences.

[10]  Daniel Ray Upper,et al.  Theory and algorithms for hidden Markov models and generalized hidden Markov models , 1998 .

[11]  James P. Crutchfield,et al.  Exact Complexity: The Spectral Decomposition of Intrinsic Computation , 2013, ArXiv.

[12]  J. Crutchfield.  The calculi of emergence: computation, dynamics and induction , 1994 .

[13]  P. Grassberger.  Toward a quantitative theory of self-generated complexity , 1986 .

[14]  James P. Crutchfield,et al.  Structure and Randomness of Continuous-Time, Discrete-Event Processes , 2017, ArXiv.

[15]  James P. Crutchfield,et al.  Predictive Rate-Distortion for Infinite-Order Markov Processes , 2016 .

[16]  Naftali Tishby,et al.  Complexity through nonextensivity , 2001, physics/0103076.

[17]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[18]  Sarah Marzen,et al.  Statistical mechanics of Monod-Wyman-Changeux (MWC) models. , 2013, Journal of molecular biology.

[19]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[20]  A. U.S.,et al.  Predictability , Complexity , and Learning , 2002 .

[21]  James P. Crutchfield,et al.  Nearly Maximally Predictive Features and Their Dimensions , 2017, Physical review. E.

[22]  Surya Ganguli,et al.  Memory traces in dynamical systems , 2008, Proceedings of the National Academy of Sciences.

[23]  James P. Crutchfield,et al.  Anatomy of a Bit: Information in a Time Series Observation , 2011, Chaos.

[24]  Susanne Still,et al.  Optimal causal inference: estimating stored information and approximating causal architecture. , 2007, Chaos.

[25]  James P Crutchfield,et al.  Time's barbed arrow: irreversibility, crypticity, and stored information. , 2009, Physical review letters.

[26]  Haim Sompolinsky,et al.  Short-term memory in orthogonal neural networks. , 2004, Physical review letters.

[27]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[28]  Naftali Tishby,et al.  Past-future information bottleneck in dynamical systems. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  James P. Crutchfield,et al.  Informational and Causal Architecture of Continuous-time Renewal Processes , 2016, 1611.01099.

[30]  Peter S. Swain,et al.  Trade-Offs and Constraints in Allosteric Sensing , 2011, PLoS Comput. Biol..