Memory and Markov Blankets

In theoretical biology, we are often interested in random dynamical systems—like the brain—that appear to model their environments. This can be formalized by appealing to the existence of a (possibly non-equilibrium) steady state, whose density preserves a conditional independence between a biological entity and its surroundings. From this perspective, the conditioning set, or Markov blanket, induces a form of vicarious synchrony between creature and world—as if one were modelling the other. However, this results in an apparent paradox. If all conditional dependencies between a system and its surroundings depend upon the blanket, how do we account for the mnemonic capacity of living systems? It might appear that any shared dependence upon past blanket states violates the independence condition, as the variables on either side of the blanket now share information not available from the current blanket state. This paper aims to resolve this paradox, and to demonstrate that conditional independence does not preclude memory. Our argument rests upon drawing a distinction between the dependencies implied by a steady state density, and the density dynamics of the system conditioned upon its configuration at a previous time. The interesting question then becomes: What determines the length of time required for a stochastic system to ‘forget’ its initial conditions? We explore this question for an example system, whose steady state density possesses a Markov blanket, through simple numerical analyses. We conclude with a discussion of the relevance for memory in cognitive systems like us.

[1]  L. Squire,et al.  Episodic memory, semantic memory, and amnesia , 1998, Hippocampus.

[2]  M. Ledoux,et al.  Analysis and Geometry of Markov Diffusion Operators , 2013 .

[3]  D. Thouless,et al.  Structure of stochastic dynamics near fixed points. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Karl J. Friston A free energy principle for a particular physics , 2019, 1906.10184.

[5]  A. Nobre,et al.  Spatial Selection of Features within Perceived and Remembered Objects , 2009, NeuroImage.

[6]  S. Amari Information geometry , 2021, Japanese Journal of Mathematics.

[7]  Thomas Parr,et al.  Deep Active Inference and Scene Construction , 2020, Frontiers in Artificial Intelligence.

[8]  Ramón Huerta,et al.  Reproducible sequence generation in random neural ensembles. , 2004, Physical review letters.

[9]  Gavin E Crooks,et al.  Measuring thermodynamic length. , 2007, Physical review letters.

[10]  H. Risken The Fokker-Planck equation : methods of solution and applications , 1985 .

[11]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[12]  Judea Pearl,et al.  Graphical Models for Probabilistic and Causal Reasoning , 1997, The Computer Science and Engineering Handbook.

[13]  Karl J. Friston,et al.  Action understanding and active inference , 2011, Biological Cybernetics.

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  W. Cannon ORGANIZATION FOR PHYSIOLOGICAL HOMEOSTASIS , 1929 .

[16]  Karl J. Friston,et al.  Neural Dynamics under Active Inference: Plausibility and Efficiency of Information Processing , 2020, Entropy.

[17]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[18]  Mark Girolami,et al.  A Unifying and Canonical Description of Measure-Preserving Diffusions , 2021, 2105.02845.

[19]  Karl J. Friston,et al.  Active inference, communication and hermeneutics , 2015, Cortex.

[20]  Eun-Jin Kim,et al.  Investigating Information Geometry in Classical and Quantum Systems through Information Length , 2018, Entropy.

[21]  Karl J. Friston,et al.  Is the Free-Energy Principle a Formal Theory of Semantics? From Variational Density Dynamics to Neural and Phenotypic Representations , 2020, Entropy.

[22]  H. Eichenbaum Time cells in the hippocampus: a new dimension for mapping memories , 2014, Nature Reviews Neuroscience.

[23]  John Duchi,et al.  Derivations for Linear Algebra and Optimization , 2016 .

[24]  Kai Ueltzhöffer,et al.  Stochastic Chaos and Markov Blankets , 2021, Entropy.

[25]  A. Baddeley Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[26]  W Grodd,et al.  Impaired procedural learning after damage to the left supplementary motor area (SMA). , 1996, Journal of neurology, neurosurgery, and psychiatry.

[27]  Michela Ottobre,et al.  Markov Chain Monte Carlo and Irreversibility , 2016 .

[28]  Adeel Razi,et al.  Parcels and particles: Markov blankets in the brain , 2020, Network Neuroscience.

[29]  Raymond J. Dolan,et al.  The anatomy of choice: dopamine and decision-making , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[30]  Karl J. Friston,et al.  Free-energy minimization in joint agent-environment systems: A niche construction perspective , 2018, Journal of theoretical biology.

[31]  Karl J. Friston,et al.  Variational ecology and the physics of sentient systems , 2019, Physics of life reviews.

[32]  Karl J. Friston,et al.  Deep temporal models and active inference , 2017, Neuroscience and biobehavioral reviews.

[33]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[34]  F. Opitz Information geometry and its applications , 2012, 2012 9th European Radar Conference.

[35]  S. Cavanagh,et al.  A Diversity of Intrinsic Timescales Underlie Neural Computations , 2020, Frontiers in Neural Circuits.

[36]  K. Spiliopoulos,et al.  Irreversible Langevin samplers and variance reduction: a large deviations approach , 2014, 1404.0105.

[37]  J. Luck,et al.  Characterising the nonequilibrium stationary states of Ornstein–Uhlenbeck processes , 2018, Journal of Physics A: Mathematical and Theoretical.

[38]  H. Risken Fokker-Planck Equation , 1996 .

[39]  R. Knight,et al.  Prefrontal modulation of visual processing in humans , 2000, Nature Neuroscience.

[40]  Tianqi Chen,et al.  Dynamical Decomposition of Markov Processes without Detailed Balance , 2013 .

[41]  P. Goldman-Rakic,et al.  Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. , 1989, Journal of neurophysiology.

[42]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[43]  Irving H. Shames,et al.  Introduction to Solid Mechanics , 1975 .

[44]  J. Krakauer,et al.  Sensory prediction errors drive cerebellum-dependent adaptation of reaching. , 2007, Journal of neurophysiology.

[45]  John M. Franchak,et al.  The development of motor behavior. , 2017, Wiley interdisciplinary reviews. Cognitive science.

[46]  J. Fuster Upper processing stages of the perception–action cycle , 2004, Trends in Cognitive Sciences.

[47]  Fuzhen Zhang The Schur complement and its applications , 2005 .

[48]  J. Fuster Unit activity in prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. , 1973, Journal of neurophysiology.

[49]  Karl J. Friston,et al.  The computational neurology of movement under active inference , 2021, Brain : a journal of neurology.

[50]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[51]  A. Nobre,et al.  Modulation of working-memory maintenance by directed attention , 2011, Neuropsychologia.

[52]  U. Seifert Stochastic thermodynamics, fluctuation theorems and molecular machines , 2012, Reports on progress in physics. Physical Society.

[53]  L. Squire Memory systems of the brain: A brief history and current perspective , 2004, Neurobiology of Learning and Memory.

[54]  Karl J. Friston,et al.  Deep Active Inference and Scene Construction , 2020, bioRxiv.

[55]  Adrian-Josue Guel-Cortez,et al.  Information Length Analysis of Linear Autonomous Stochastic Processes , 2020, Entropy.

[56]  C. Marsden,et al.  PROCEDURAL MEMORY AND NEUROLOGICAL DISEASE , 1992 .

[57]  Karl J. Friston Life as we know it , 2013, Journal of The Royal Society Interface.

[58]  Grigorios A. Pavliotis,et al.  Bayesian mechanics for stationary processes , 2021, Proceedings of the Royal Society A.

[59]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[60]  G. Pavliotis Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations , 2014 .

[61]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[62]  Kai Ueltzhöffer,et al.  Deep active inference , 2017, Biological Cybernetics.

[63]  A. Nobre,et al.  Attentional modulation of object representations in working memory. , 2007, Cerebral cortex.

[64]  Karl J. Friston,et al.  Population dynamics under the Laplace assumption , 2009, NeuroImage.

[65]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[66]  Karl J. Friston,et al.  Prefrontal Computation as Active Inference , 2019, Cerebral cortex.

[67]  Karl J. Friston,et al.  Markov blankets, information geometry and stochastic thermodynamics , 2019, Philosophical Transactions of the Royal Society A.

[68]  Karl J. Friston,et al.  Working memory, attention, and salience in active inference , 2017, Scientific Reports.

[69]  PascucciValerio,et al.  The Helmholtz-Hodge Decomposition—A Survey , 2013 .

[70]  M. Molinari,et al.  Cerebellum and procedural learning: evidence from focal cerebellar lesions. , 1997, Brain : a journal of neurology.

[71]  Karl J. Friston,et al.  Active Inference and Learning in the Cerebellum , 2016, Neural Computation.

[72]  Karl J. Friston,et al.  The computational pharmacology of oculomotion , 2019, Psychopharmacology.