A probabilistic interpretation of PID controllers using active inference

In the past few decades, probabilistic interpretations of brain functions have become widespread in cognitive science and neuroscience. The Bayesian brain hypothesis, predictive coding, the free energy principle and active inference are increasingly popular theories of cognitive functions that claim to unify understandings of life and cognition within general mathematical frameworks derived from information and control theory, statistical physics and machine learning. The connections between information and control theory have been discussed since the 1950’s by scientists like Shannon and Kalman and have recently risen to prominence in modern stochastic optimal control theory. However, the implications of the confluence of these two theoretical frameworks for the biological sciences have been slow to emerge. Here we argue that if the active inference proposal is to be taken as a general process theory for biological systems, we need to consider how existing control theoretical approaches to biological systems relate to it. In this work we will focus on PID (Proportional-Integral-Derivative) controllers, one of the most common types of regulators employed in engineering and more recently used to explain behaviour in biological systems, e.g. chemotaxis in bacteria and amoebae or robust adaptation in biochemical networks. Using active inference, we derive a probabilistic interpretation of PID controllers, showing how they can fit a more general theory of life and cognition under the principle of (variational) free energy minimisation under simple linear generative models.

[1]  Emanuel Todorov,et al.  General duality between optimal control and estimation , 2008, 2008 47th IEEE Conference on Decision and Control.

[2]  Anil K. Seth,et al.  The cybernetic Bayesian brain: from interoceptive inference to sensorimotor contingencies , 2014 .

[3]  A. Clark Radical predictive processing , 2015 .

[4]  Christopher L. Buckley,et al.  An active inference implementation of phototaxis , 2017, ECAL.

[5]  Karl J. Friston,et al.  DEM: A variational treatment of dynamic systems , 2008, NeuroImage.

[6]  Karl J. Friston,et al.  Action and behavior: a free-energy formulation , 2010, Biological Cybernetics.

[7]  Karl J. Friston Life as we know it , 2013, Journal of The Royal Society Interface.

[8]  Norbert Wiener,et al.  Cybernetics, or control and communication in the animal and the machine, 2nd ed. , 1961 .

[9]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[10]  W. Ashby,et al.  An Introduction to Cybernetics , 1957 .

[11]  C. Carver,et al.  Attention and Self-Regulation: A Control-Theory Approach to Human Behavior , 1981 .

[12]  Liu Yang,et al.  Positive feedback may cause the biphasic response observed in the chemoattractant-induced response of Dictyostelium cells , 2006, Syst. Control. Lett..

[13]  A. Borst Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[14]  Karl J. Friston The free-energy principle: a rough guide to the brain? , 2009, Trends in Cognitive Sciences.

[15]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[16]  Karl J. Friston,et al.  Action understanding and active inference , 2011, Biological Cybernetics.

[17]  A. Clark Being There: Putting Brain, Body, and World Together Again , 1996 .

[18]  Simon McGregor,et al.  The free energy principle for action and perception: A mathematical review , 2017, 1705.09156.

[19]  F. H. Adler Cybernetics, or Control and Communication in the Animal and the Machine. , 1949 .

[20]  A. Jazwinski Stochastic Processes and Filtering Theory , 1970 .

[21]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[22]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[23]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[24]  Karl J. Friston,et al.  Active inference and epistemic value (vol 6, pg 187, 2015) , 2016 .

[25]  Viktor Mikhaĭlovich Glushkov,et al.  An Introduction to Cybernetics , 1957, The Mathematical Gazette.

[26]  M. Frank,et al.  A control theoretic model of adaptive behavior in dynamic environments , 2017, bioRxiv.

[27]  Rafal Bogacz,et al.  A tutorial on the free-energy framework for modelling perception and learning , 2017, Journal of mathematical psychology.

[28]  Karl J. Friston What Is Optimal about Motor Control? , 2011, Neuron.

[29]  Karl J. Friston Hierarchical Models in the Brain , 2008, PLoS Comput. Biol..

[30]  W. Ashby,et al.  Requisite Variety and Its Implications for the Control of Complex Systems , 1991 .

[31]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[32]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[33]  M. Hoagland,et al.  Feedback Systems An Introduction for Scientists and Engineers SECOND EDITION , 2015 .

[34]  J. Doyle,et al.  Robust perfect adaptation in bacterial chemotaxis through integral feedback control. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Karl J. Friston,et al.  Reinforcement Learning or Active Inference? , 2009, PloS one.

[36]  Karl J. Friston,et al.  Active inference, sensory attenuation and illusions , 2013, Cognitive Processing.

[37]  Karl J. Friston,et al.  A free energy principle for the brain , 2006, Journal of Physiology-Paris.

[38]  W. T. Powers Behavior, the control of perception , 1973 .

[39]  R. E. Kalman,et al.  Contributions to the Theory of Optimal Control , 1960 .

[40]  Karl Johan Åström,et al.  PID Controllers: Theory, Design, and Tuning , 1995 .

[41]  Karl J. Friston,et al.  Active inference and epistemic value , 2015, Cognitive neuroscience.

[42]  Brian Ingalls,et al.  Considerations for using integral feedback control to construct a perfectly adapting synthetic gene network. , 2010, Journal of theoretical biology.

[43]  M. Nour Surfing Uncertainty: Prediction, Action, and the Embodied Mind. , 2017, British Journal of Psychiatry.

[44]  J. Hohwy The Predictive Mind , 2013 .