Bayesian mechanics for stationary processes

This paper develops a Bayesian mechanics for adaptive systems. Firstly, we model the interface between a system and its environment with a Markov blanket. This affords conditions under which states internal to the blanket encode information about external states. Second, we introduce dynamics and represent adaptive systems as Markov blankets at steady state. This allows us to identify a wide class of systems whose internal states appear to infer external states, consistent with variational inference in Bayesian statistics and theoretical neuroscience. Finally, we partition the blanket into sensory and active states. It follows that active states can be seen as performing active inference and well-known forms of stochastic control (such as PID control), which are prominent formulations of adaptive behaviour in theoretical biology and engineering.

[1]  J. Elgin The Fokker-Planck Equation: Methods of Solution and Applications , 1984 .

[2]  Karl J. Friston,et al.  The graphical brain: Belief propagation and active inference , 2017, Network Neuroscience.

[3]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[4]  Rafal Bogacz,et al.  A tutorial on the free-energy framework for modelling perception and learning , 2017, Journal of mathematical psychology.

[5]  S. Mitter,et al.  Toward a theory of nonlinear stochastic realization , 1982 .

[6]  André Elisseeff,et al.  Using Markov Blankets for Causal Structure Learning , 2008, J. Mach. Learn. Res..

[7]  Eero P. Simoncelli,et al.  Spike-triggered neural characterization. , 2006, Journal of vision.

[8]  Karl J. Friston,et al.  Some Interesting Observations on the Free Energy Principle , 2020, Entropy.

[9]  Raquel Oliveira Prates,et al.  Active Inference: First International Workshop, IWAI 2020, Co-located with ECML/PKDD 2020, Ghent, Belgium, September 14, 2020, Proceedings , 2020, IWAI.

[10]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[11]  Karl J. Friston,et al.  The computational neurology of movement under active inference , 2021, Brain : a journal of neurology.

[12]  U. Haussmann,et al.  TIME REVERSAL OF DIFFUSIONS , 1986 .

[13]  G. Pavliotis,et al.  Optimal Non-reversible Linear Drift for the Convergence to Equilibrium of a Diffusion , 2012, 1212.0876.

[14]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[15]  D. Wolpert Minimal entropy production rate of interacting systems , 2020, New Journal of Physics.

[16]  G. Roberts,et al.  A piecewise deterministic scaling limit of Lifted Metropolis-Hastings in the Curie-Weiss model , 2015, 1509.00302.

[17]  Mikhail Kryachkov,et al.  Finite-time stabilization of an integrator chain using only signs of the state variables , 2010, 2010 11th International Workshop on Variable Structure Systems (VSS).

[18]  G. Pavliotis Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations , 2014 .

[19]  Robert Graham,et al.  Covariant formulation of non-equilibrium statistical thermodynamics , 1977 .

[20]  I. Prigogine,et al.  Formative Processes. (Book Reviews: Self-Organization in Nonequilibrium Systems. From Dissipative Structures to Order through Fluctuations) , 1977 .

[21]  Karl J. Friston Life as we know it , 2013, Journal of The Royal Society Interface.

[22]  Manuel Baltieri,et al.  PID Control as a Process of Active Inference with Linear Generative Models † , 2019, Entropy.

[23]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[24]  H. Haken Synergetics: an Introduction, Nonequilibrium Phase Transitions and Self-organization in Physics, Chemistry, and Biology , 1977 .

[25]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[26]  Karl J. Friston,et al.  Variational free energy and the Laplace approximation , 2007, NeuroImage.

[27]  Kai Ueltzhöffer,et al.  Deep active inference , 2017, Biological Cybernetics.

[28]  B. Øksendal Stochastic differential equations : an introduction with applications , 1987 .

[29]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[30]  Susanne Still,et al.  The thermodynamics of prediction , 2012, Physical review letters.

[31]  Jörn Dunkel,et al.  Improved bounds on entropy production in living systems , 2021, Proceedings of the National Academy of Sciences.

[32]  How particular is the physics of the free energy principle? , 2021, Physics of life reviews.

[33]  Robert J. Kosinski,et al.  A Literature Review on Reaction Time Kinds of Reaction Time Experiments , 2012 .

[34]  The Fokker Planck Equation Methods Of Solution And Applications Springer Series In Synergetics , 2020 .

[35]  Karl J. Friston,et al.  Generalised Filtering , 2010 .

[36]  Ichiro Aoki,et al.  Entropy production in living systems : from organisms to ecosystems , 1995 .

[37]  Luc Rey-Bellet,et al.  Open classical systems , 2006 .

[38]  J. Doob,et al.  The Brownian Movement and Stochastic Equations , 1942 .

[39]  M. Yor DIFFUSIONS, MARKOV PROCESSES AND MARTINGALES: Volume 2: Itô Calculus , 1989 .

[40]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[41]  Kate Jeffery,et al.  On the Statistical Mechanics of Life: Schrödinger Revisited , 2019, Entropy.

[42]  G. Uhlenbeck,et al.  On the Theory of the Brownian Motion II , 1945 .

[43]  Liping Wang,et al.  Large-Scale Cortical Networks for Hierarchical Prediction and Prediction Error in the Primate Brain , 2018, Neuron.

[44]  Karl J. Friston,et al.  Predictive coding under the free-energy principle , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[45]  Stefano Soatto,et al.  Stochastic Gradient Descent Performs Variational Inference, Converges to Limit Cycles for Deep Networks , 2017, 2018 Information Theory and Applications Workshop (ITA).

[46]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[47]  Hana El-Samad,et al.  Design and analysis of a Proportional-Integral-Derivative controller with biological molecules , 2018, bioRxiv.

[48]  Susanne Still,et al.  Thermodynamic Cost and Benefit of Memory. , 2017, Physical review letters.

[49]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[50]  Adeel Razi,et al.  Parcels and particles: Markov blankets in the brain , 2020, Network Neuroscience.

[51]  Karl J. Friston Variational filtering , 2008, NeuroImage.

[52]  Alejandro Marambio-Tapia,et al.  Off , 2020, Definitions.

[53]  M. James,et al.  The generalised inverse , 1978, The Mathematical Gazette.

[54]  Jürg Kohlas,et al.  Handbook of Defeasible Reasoning and Uncertainty Management Systems , 2000 .

[55]  David H. Wolpert Uncertainty relations and fluctuation theorems for Bayes nets , 2020, Physical review letters.

[56]  C. Mathys,et al.  Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning , 2013, Neuron.

[57]  Karl J. Friston,et al.  Action and behavior: a free-energy formulation , 2010, Biological Cybernetics.

[58]  Karl J. Friston What Is Optimal about Motor Control? , 2011, Neuron.

[60]  K. Spiliopoulos,et al.  Irreversible Langevin samplers and variance reduction: a large deviations approach , 2014, 1404.0105.

[61]  M. L. Eaton Multivariate statistics : a vector space approach , 1985 .

[62]  A. Frazho On stochastic realization theory , 1982 .

[63]  M. Pavon,et al.  On the nonlinear stochastic realization problem , 1989 .

[64]  Mariana Gómez-Schiavon,et al.  Design and Analysis of a Proportional-Integral-Derivative Controller with Biological Molecules. , 2019, Cell systems.

[65]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[66]  A. Doucet,et al.  The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method , 2015, 1510.02451.

[67]  Karl J. Friston,et al.  Predictions not commands: active inference in the motor system , 2012, Brain Structure and Function.

[68]  M. J. Friedlander,et al.  The time course and amplitude of EPSPs evoked at synapses between pairs of CA3/CA1 neurons in the hippocampal slice , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[69]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[70]  Kai Ueltzhoffer On the thermodynamics of prediction under dissipative adaptation , 2020, 2009.04006.

[71]  Gregory L. Eyink,et al.  Hydrodynamics and fluctuations outside of local equilibrium: Driven diffusive systems , 1996 .

[72]  Mark Girolami,et al.  A Unifying and Canonical Description of Measure-Preserving Diffusions , 2021, 2105.02845.

[73]  Karl J. Friston,et al.  The Markov blankets of life: autonomy, active inference and the free energy principle , 2018, Journal of The Royal Society Interface.

[74]  H. Risken Fokker-Planck Equation , 1996 .

[75]  Giovanni Pezzulo,et al.  An Active Inference view of cognitive control , 2012, Front. Psychology.

[76]  Tianqi Chen,et al.  A Complete Recipe for Stochastic Gradient MCMC , 2015, NIPS.

[77]  Karl J. Friston,et al.  The emergence of synchrony in networks of mutually inferring neurons , 2019, Scientific Reports.

[78]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[79]  Matthew J. Beal Variational algorithms for approximate Bayesian inference , 2003 .

[80]  Xiaowu Dai,et al.  On Large Batch Training and Sharp Minima: A Fokker–Planck Perspective , 2020, Journal of Statistical Theory and Practice.

[81]  Michela Ottobre,et al.  Markov Chain Monte Carlo and Irreversibility , 2016 .

[82]  Robert Marsland,et al.  Statistical Physics of Adaptation , 2014, 1412.1875.

[83]  Jordan M. Horowitz,et al.  Thermodynamics with Continuous Information Flow , 2014, 1402.3276.

[84]  G. A. Pavliotis,et al.  Mean Field Limits for Interacting Diffusions with Colored Noise: Phase Transitions and Spectral Numerical Methods , 2019, Multiscale Model. Simul..

[85]  H. Haken,et al.  Synergetics , 1988, IEEE Circuits and Devices Magazine.

[86]  Andrew M. Stuart,et al.  Convergence of Numerical Time-Averaging and Stationary Measures via Poisson Equations , 2009, SIAM J. Numer. Anal..

[87]  Michael J. Berry,et al.  The Neural Code of the Retina , 1999, Neuron.

[88]  Karl J. Friston A free energy principle for a particular physics , 2019, 1906.10184.

[89]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[90]  Robert W. Spekkens,et al.  Reconstruction of Gaussian quantum mechanics from Liouville mechanics with an epistemic restriction , 2011, 1111.5057.

[91]  R. Zemel,et al.  Inference and computation with population codes. , 2003, Annual review of neuroscience.

[92]  Thomas Parr,et al.  The computational neurology of active vision , 2019 .

[93]  Judea Pearl,et al.  Graphical Models for Probabilistic and Causal Reasoning , 1997, The Computer Science and Engineering Handbook.

[94]  G. Picci,et al.  Linear Stochastic Systems , 2015 .

[95]  Tony Roskilly,et al.  Marine systems identification, modeling and control , 2015 .

[96]  R. Ramaswamy,et al.  Generalized synchrony of coupled stochastic processes with multiplicative noise. , 2016, Physical review. E.

[97]  Karl J. Friston,et al.  DEM: A variational treatment of dynamic systems , 2008, NeuroImage.

[98]  P Ao,et al.  LETTER TO THE EDITOR: Potential in stochastic differential equations: novel construction , 2004 .

[99]  Corrado Pezzato,et al.  A Novel Adaptive Controller for Robot Manipulators Based on Active Inference , 2020, IEEE Robotics and Automation Letters.

[100]  J. Doyle,et al.  Robust perfect adaptation in bacterial chemotaxis through integral feedback control. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[101]  Kai Ueltzhöffer,et al.  Stochastic Chaos and Markov Blankets , 2021, Entropy.

[102]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[103]  H. Qian A decomposition of irreversible diffusion processes without detailed balance , 2012, 1204.6496.

[104]  Gordon Cheng,et al.  An Empirical Study of Active Inference on a Humanoid Robot , 2021, IEEE Transactions on Cognitive and Developmental Systems.

[105]  Jeremy L. England,et al.  Statistical physics of self-replication. , 2012, The Journal of chemical physics.

[106]  Konstantin Zimenko,et al.  Finite-time and fixed-time stabilization for integrator chain of arbitrary order* , 2018, 2018 European Control Conference (ECC).

[107]  Karl J. Friston Hierarchical Models in the Brain , 2008, PLoS Comput. Biol..

[108]  A. Goldbeter Dissipative structures in biological systems: bistability, oscillations, spatial patterns and waves , 2018, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[109]  Karl J. Friston,et al.  Markov blankets, information geometry and stochastic thermodynamics , 2019, Philosophical Transactions of the Royal Society A.

[110]  V. Araújo Random Dynamical Systems , 2006, math/0608162.

[111]  Karl J. Friston,et al.  A Multi-scale View of the Emergent Complexity of Life: A Free-Energy Proposal , 2019, Evolution, Development and Complexity.

[112]  G. Crooks,et al.  Marginal and conditional second laws of thermodynamics , 2016, EPL (Europhysics Letters).

[113]  Gordon Cheng,et al.  Robot self/other distinction: active inference meets neural networks learning in a mirror , 2020, ECAI.

[114]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[115]  G. Picci,et al.  Linear Stochastic Systems: A Geometric Approach to Modeling, Estimation and Identification , 2016 .

[116]  D. Cumin,et al.  Generalising the Kuramoto Model for the study of Neuronal Synchronisation in the Brain , 2007 .

[117]  Ryota Kanai,et al.  A Technical Critique of Some Parts of the Free Energy Principle , 2021, Entropy.

[118]  Yasser Roudi,et al.  Learning and inference in a nonequilibrium Ising model with hidden nodes. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[119]  Maxim Raginsky,et al.  Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit , 2019, ArXiv.

[120]  Beren Millidge,et al.  Deep Active Inference as Variational Policy Gradients , 2019, Journal of Mathematical Psychology.

[121]  S. Shreve,et al.  Stochastic differential equations , 1955, Mathematical Proceedings of the Cambridge Philosophical Society.

[122]  G. Picci,et al.  Realization Theory for Multivariate Stationary Gaussian Processes , 1985 .

[123]  Pablo Lanillos,et al.  End-to-End Pixel-Based Deep Active Inference for Body Perception and Action , 2020, 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[124]  Simon McGregor,et al.  The free energy principle for action and perception: A mathematical review , 2017, 1705.09156.

[125]  Karl J. Friston,et al.  Deep Active Inference and Scene Construction , 2020, bioRxiv.

[126]  P. Fearnhead,et al.  The Zig-Zag process and super-efficient sampling for Bayesian analysis of big data , 2016, The Annals of Statistics.

[127]  Magnus T. Koudahl,et al.  A Worked Example of Fokker-Planck-Based Active Inference , 2020, IWAI.