The Neural Costs of Optimal Control

Optimal control entails combining probabilities and utilities. However, for most practical problems, probability densities can be represented only approximately. Choosing an approximation requires balancing the benefits of an accurate approximation against the costs of computing it. We propose a variational framework for achieving this balance and apply it to the problem of how a neural population code should optimally represent a distribution under resource constraints. The essence of our analysis is the conjecture that population codes are organized to maximize a lower bound on the log expected utility. This theory can account for a plethora of experimental data, including the reward-modulation of sensory receptive fields, GABAergic effects on saccadic movements, and risk aversion in decisions under uncertainty.

[1]  M. Bear,et al.  Reward Timing in the Primary Visual Cortex , 2006, Science.

[2]  Maneesh Sahani,et al.  A Biologically Plausible Algorithm for Reinforcement-shaped Representational Learning , 2003, NIPS.

[3]  Christian K. Machens,et al.  Testing the Efficiency of Sensory Coding with Optimal Stimulus Ensembles , 2005, Neuron.

[4]  E. Jaynes.  On the rationale of maximum-entropy methods , 1982, Proceedings of the IEEE.

[5]  P. Montague,et al.  Efficient statistics, common currencies and the problem of reward-harvesting , 2007, Trends in Cognitive Sciences.

[6]  William B. Levy,et al.  Energy Efficient Neural Codes , 1996, Neural Computation.

[7]  H. Simon,et al.  Models of Bounded Rationality: Empirically Grounded Economic Reason , 1997 .

[8]  Alexandre Pouget,et al.  Probabilistic Interpretation of Population Codes , 1996, Neural Computation.

[9]  David Barber,et al.  Variational methods for Reinforcement Learning , 2010, AISTATS.

[10]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[11]  I. Deary,et al.  Visual information processing during controlled hypoglycaemia in humans. , 1996, Brain : a journal of neurology.

[12]  Leif H. Finkel,et al.  A Neural Implementation of the Kalman Filter , 2009, NIPS.

[13]  Naomi H. Feldman,et al.  Performing Bayesian Inference with Exemplar Models , 2008 .

[14]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[15]  D. Ferster,et al.  The contribution of noise to contrast invariance of orientation tuning in cat visual cortex. , 2000, Science.

[16]  R. L. Valois,et al.  The orientation and direction selectivity of cells in macaque visual cortex , 1982, Vision Research.

[17]  Robert M. McPeek,et al.  Deficits in saccade target selection after inactivation of superior colliculus , 2004, Nature Neuroscience.

[18]  Joshua B. Tenenbaum,et al.  Perceptual Multistability as Markov Chain Monte Carlo Inference , 2009, NIPS.

[19]  John T Serences,et al.  Value-Based Modulations in Human Visual Cortex , 2008, Neuron.

[20]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[21]  P. E. Gold,et al.  Role of glucose in regulating the brain and cognition. , 1995, The American journal of clinical nutrition.

[22]  L. J. Savage,et al.  The Foundations of Statistics , 1955 .

[23]  Thomas L. Griffiths,et al.  Neural Implementation of Hierarchical Bayesian Inference by Importance Sampling , 2009, NIPS.

[24]  R. Freeman,et al.  Orientation selectivity in the cat's striate cortex is invariant with stimulus contrast , 2004, Experimental Brain Research.

[25]  Karl J. Friston.  The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[26]  Rajesh P. N. Rao.  Bayesian Computation in Recurrent Neural Circuits , 2004, Neural Computation.