The free-energy principle: a unified brain theory?

A free-energy principle has been proposed recently that accounts for action, perception and learning. This Review looks at some key brain theories in the biological (for example, neural Darwinism) and physical (for example, information theory and optimal control theory) sciences from the free-energy perspective. Crucially, one key theme runs through each of these theories — optimization. Furthermore, if we look closely at what is optimized, the same quantity keeps emerging, namely value (expected reward, expected utility) or its complement, surprise (prediction error, expected cost). This is the quantity that is optimized under the free-energy principle, which suggests that several global brain theories might be unified within a free-energy framework.

[1]  W. Ashby,et al.  Principles of the self-organizing dynamic system. , 1947, The Journal of general psychology.

[2]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[3]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[5]  R. Gregory,et al.  Perceptual illusions and brain models , 1968, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[6]  R. Feynman.  Statistical Mechanics, A Set of Lectures , 1972 .

[7]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[8]  Richard Phillips Feynman,et al.  Statistical Mechanics: A Set of Lectures , 1972 .

[9]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[10]  B. Efron,et al.  Stein's Estimation Rule and Its Competitors- An Empirical Bayes Approach , 1973 .

[11]  D. S. Jones,et al.  Elementary information theory , 1979 .

[12]  R. Gregory.  Perceptions as hypotheses. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[13]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[14]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  Geoffrey E. Hinton,et al.  Parallel visual computation , 1983, Nature.

[16]  L. Optican,et al.  Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. , 1987, Journal of neurophysiology.

[17]  H. Spitzer,et al.  Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. I. Response characteristics. , 1987, Journal of neurophysiology.

[18]  I. Introduction , 1988, Agricultural Labour in Indian Society.

[19]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[20]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[21]  R. Kass,et al.  Approximate Bayesian Inference in Conditionally Independent Hierarchical Models (Parametric Empirical Bayes Models) , 1989 .

[22]  R Linsker,et al.  Perceptual neural organization: some approaches based on network models and information theory. , 1990, Annual review of neuroscience.

[23]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[24]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[25]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[26]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[27]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[28]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[29]  Mitsuo Kawato,et al.  A forward-inverse optics model of reciprocal connections between visual cortical areas , 1993 .

[30]  G. Edelman.  Neural Darwinism: Selection and reentrant signaling in higher brain function , 1993, Neuron.

[31]  H. Crauel,et al.  Attractors for random dynamical systems , 1994 .

[32]  W J Freeman,et al.  Characterization of state transitions in spatially distributed, chaotic, nonlinear, dynamical systems in cerebral cortex , 1994, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[33]  Karl J. Friston,et al.  Value-dependent selection in the brain: Simulation in a synthetic neural model , 1994, Neuroscience.

[34]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[35]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[36]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[37]  D. Mackay.  Free energy minimisation algorithm for decoding and cryptanalysis , 1995 .

[38]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[39]  Peter Dayan,et al.  Bee foraging in uncertain environments using predictive hebbian learning , 1995, Nature.

[40]  Arantxa Etxeverria.  The Origins of Order , 1993 .

[41]  R. Desimone,et al.  Neural mechanisms for visual memory and their role in attention. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[42]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[43]  M. Bear,et al.  Metaplasticity: the plasticity of synaptic plasticity , 1996, Trends in Neurosciences.

[44]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[45]  T. Sanger,et al.  Probability density estimation for the interpretation of neural population codes. , 1996, Journal of neurophysiology.

[46]  L. Abbott,et al.  Synaptic Depression and Cortical Gain Control , 1997, Science.

[47]  M. Mesulam,et al.  From sensation to cognition. , 1998, Brain : a journal of neurology.

[48]  H. Maturana,et al.  De máquinas y seres vivos , 1998 .

[49]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[50]  A Treisman,et al.  Feature binding, attention and object perception. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[51]  R. Guillery,et al.  On the actions that one nerve cell can have on another: distinguishing "drivers" from "modulators". , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[52]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[53]  Alexandre Pouget,et al.  Probabilistic Interpretation of Population Codes , 1996, Neural Computation.

[54]  Michael I. Jordan,et al.  Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. , 1998, Journal of neurophysiology.

[55]  Karl J. Friston,et al.  The Relationship Between Synchronization Among Neuronal Populations and Their Mean Activity Levels , 1999, Neural Computation.

[56]  Zoubin Ghahramani,et al.  A Unifying Review of Linear Gaussian Models , 1999, Neural Computation.

[57]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[58]  J. Cohen,et al.  The role of locus coeruleus in the regulation of cognitive performance. , 1999, Science.

[59]  T. Sejnowski,et al.  Natural patterns of activity and long-term synaptic plasticity , 2000, Current Opinion in Neurobiology.

[60]  C. Voigt,et al.  Rational evolutionary design: the theory of in vitro protein evolution. , 2000, Advances in protein chemistry.

[61]  Karl J. Friston,et al.  The labile brain. III. Transients and spatio-temporal receptive fields. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[62]  S. Laughlin.  Efficiency and complexity in neural coding. , 2008, Novartis Foundation symposium.

[63]  G D Lewen,et al.  Neural coding of naturalistic motion stimuli , 2001, Network.

[64]  F. Knobloch.  Altruism and the hypothesis of meta-selection in human evolution. , 2001, The Journal of the American Academy of Psychoanalysis.

[65]  John J. Foxe,et al.  Determinants and mechanisms of attentional modulation of neural processing. , 2001, Frontiers in bioscience : a journal and virtual library.

[66]  I. Tsuda.  Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems. , 2001, The Behavioral and brain sciences.

[67]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[68]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[69]  Junichiro Yoshimoto,et al.  Control of exploitation-exploration meta-parameter in reinforcement learning , 2002, Neural Networks.

[70]  A. U.S.,et al.  Predictability , Complexity , and Learning , 2002 .

[71]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[72]  Paul Schrater,et al.  Shape perception reduces activity in human primary visual cortex , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[73]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[74]  Paul F. M. J. Verschure,et al.  Environmentally mediated synergy between perception and behaviour in mobile robots , 2003, Nature.

[75]  Colin Camerer.  Behavioural studies of strategic thinking in games , 2003, Trends in Cognitive Sciences.

[76]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[77]  Denis J. Evans,et al.  A non-equilibrium free energy theorem for deterministic systems , 2003 .

[78]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[79]  W. A. Phillips,et al.  Where the rubber meets the road: The importance of implementation , 2003, Behavioral and Brain Sciences.

[80]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[81]  Shin Ishii,et al.  Bayesian representation learning in the cortex regulated by acetylcholine , 2004, Neural Networks.

[82]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[83]  G. Pareti,et al.  Does the brain oscillate? The dispute on neuronal synchronization , 2004, Neurological Sciences.

[84]  T. Frank.  Nonlinear Fokker-Planck Equations: Fundamentals and Applications , 2004 .

[85]  D. Mumford.  On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[86]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[87]  E. Beghi,et al.  Basic and clinical research on amyotrophic lateral sclerosis and other motor neuron disorders in Italy: recent findings and achievements from a network of laboratories , 2004, Neurological Sciences.

[88]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[89]  Viktor K. Jirsa,et al.  A theoretical model of phase transitions in the human brain , 1994, Biological Cybernetics.

[90]  May-Britt Moser,et al.  Place cells, spatial maps and the population code for memory , 2005, Current Opinion in Neurobiology.

[91]  M G Paulin,et al.  Evolution of the cerebellum as a neuronal machine for Bayesian state estimation , 2005, Journal of neural engineering.

[92]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[93]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[94]  P. Dayan,et al.  Dopamine, uncertainty and TD learning , 2005, Behavioral and Brain Functions.

[95]  Michael Breakspear,et al.  Dynamics of a neural system with a multiscale architecture , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[96]  A. Angelucci,et al.  Contribution of feedforward, lateral and feedback connections to the classical receptive field center and extra-classical receptive field surround of primate V1 neurons. , 2006, Progress in brain research.

[97]  T. Womelsdorf,et al.  Neuronal coherence during selective attentional processing and sensory–motor integration , 2006, Journal of Physiology-Paris.

[98]  K. Berridge.  The debate over dopamine’s role in reward: the case for incentive salience , 2007, Psychopharmacology.

[99]  P. Redgrave,et al.  The short-latency dopamine signal: a role in discovering novel actions? , 2006, Nature Reviews Neuroscience.

[100]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[101]  Alessandro Treves,et al.  Autoassociative memory retrieval and spontaneous activity bumps in small-world networks of integrate-and-fire neurons , 2005, Journal of Physiology-Paris.

[102]  Karl J. Friston,et al.  A free energy principle for the brain , 2006, Journal of Physiology-Paris.

[103]  J. Maunsell,et al.  Feature-based attention in visual cortex , 2006, Trends in Neurosciences.

[104]  K. Doya,et al.  The computational neurobiology of learning and reward , 2006, Current Opinion in Neurobiology.

[105]  S. Bressler,et al.  Operational principles of neurocognitive networks. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[106]  D. Durstewitz,et al.  Beyond bistability: Biophysics and temporal dynamics of working memory , 2006, Neuroscience.

[107]  Daniel M Wolpert,et al.  Computational principles of sensorimotor control that minimize uncertainty and variability , 2007, The Journal of physiology.

[108]  Karl J. Friston,et al.  Variational free energy and the Laplace approximation , 2007, NeuroImage.

[109]  Gerhard Werner,et al.  Brain dynamics across levels of organization , 2007, Journal of Physiology-Paris.

[110]  J. Krakauer,et al.  Sensory prediction errors drive cerebellum-dependent adaptation of reaching. , 2007, Journal of neurophysiology.

[111]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[112]  H. Haken,et al.  Intentionality in non-equilibrium systems? The functional aspects of self-organized pattern formation , 2007 .

[113]  S. Grossberg.  Towards a unified theory of neocortex: laminar cortical circuits for vision and cognition. , 2007, Progress in brain research.

[114]  Karl J. Friston.  Hierarchical Models in the Brain , 2008, PLoS Comput. Biol..

[115]  S. Grossberg,et al.  Spikes, synchrony, and attentive learning by laminar thalamocortical circuits , 2006, Brain Research.

[116]  Michael W. Spratling.  Predictive coding as a model of biased competition in visual attention , 2008, Vision Research.

[117]  David Applebaum,et al.  Probability and Information: An Integrated Approach , 2008 .

[118]  Gregory T. Huang.  Essence of thought , 2008 .

[119]  T. Paus,et al.  Why do many psychiatric disorders emerge during adolescence? , 2008, Nature Reviews Neuroscience.

[120]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.

[121]  L. L. Bologna,et al.  Self-organization and neuronal avalanches in networks of dissociated cortical neurons , 2008, Neuroscience.

[122]  Gilles Laurent,et al.  Transient Dynamics for Neural Processing , 2008, Science.

[123]  R. Desimone,et al.  The Effects of Visual Stimulation and Selective Visual Attention on Rhythmic Neuronal Synchronization in Macaque Area V4 , 2008, The Journal of Neuroscience.

[124]  D. Heeger,et al.  The Normalization Model of Attention , 2009, Neuron.

[125]  Giorgio F. Gilestro,et al.  Widespread Changes in Synaptic Markers as a Function of Sleep and Wakefulness in Drosophila , 2009, Science.

[126]  V. Pande,et al.  On the application of statistical physics to evolutionary biology. , 2009, Journal of theoretical biology.

[127]  Karl J. Friston,et al.  Reinforcement Learning or Active Inference? , 2009, PloS one.

[128]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[129]  Karl J. Friston,et al.  Cortical circuits for perceptual inference , 2009, Neural Networks.

[130]  ChrisD . Frith,et al.  Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia , 2009, Nature Reviews Neuroscience.

[131]  Karl J. Friston,et al.  Dynamic Causal Modeling of the Response to Frequency Deviants , 2009, Journal of neurophysiology.

[132]  Edward T. Bullmore,et al.  Broadband Criticality of Human Brain Network Synchronization , 2009, PLoS Comput. Biol..

[133]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .