Efficient Neural Codes under Metabolic Constraints

Neural codes are inevitably shaped by various kinds of biological constraints, \emph{e.g.} noise and metabolic cost. Here we formulate a coding framework which explicitly deals with noise and the metabolic costs associated with the neural representation of information, and analytically derive the optimal neural code for monotonic response functions and arbitrary stimulus distributions. For a single neuron, the theory predicts a family of optimal response functions depending on the metabolic budget and noise characteristics. Interestingly, the well-known histogram equalization solution can be viewed as a special case when metabolic resources are unlimited. For a pair of neurons, our theory suggests that under more severe metabolic constraints, ON-OFF coding is an increasingly more efficient coding scheme compared to ON-ON or OFF-OFF. The advantage could be as large as one-fold, substantially larger than the previous estimation. Some of these predictions could be generalized to the case of large neural populations. In particular, these analytical results may provide a theoretical basis for the predominant segregation into ON- and OFF-cells in early visual processing areas. Overall, we provide a unified framework for optimal neural codes with monotonic tuning curves in the brain, and makes predictions that can be directly tested with physiology experiments.

[1]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[2]  Zhuo Wang,et al.  "Optimal Neural Tuning Curves for Arbitrary Stimulus Distributions: Discrimax, Infomax and Minimum $L_p$ Loss" , 2012, NIPS.

[3]  J. C. Middlebrooks,et al.  Location Coding by Opponent Neural Populations in the Auditory Cortex , 2005, PLoS biology.

[4]  H. Sompolinsky,et al.  Benefits of Pathway Splitting in Sensory Coding , 2014, The Journal of Neuroscience.

[5]  W. Bialek,et al.  Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents , 1995, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[6]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[7]  Joseph J Atick,et al.  Could information theory provide an ecological theory of sensory processing? , 2011, Network.

[8]  D. Snodderly,et al.  High response reliability of neurons in primary visual cortex (V1) of alert, trained monkeys. , 2006, Cerebral cortex.

[9]  Andrew M. Clark,et al.  Stimulus onset quenches neural variability: a widespread cortical phenomenon , 2010, Nature Neuroscience.

[10]  M. Carandini Amplification of Trial-to-Trial Response Variability by Neurons in Visual Cortex , 2004, PLoS biology.

[11]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[12]  David McAlpine,et al.  Optimal neural population coding of an auditory spatial cue , 2004, Nature.

[13]  Heinz Wässle,et al.  Parallel processing in the mammalian retina , 2004, Nature Reviews Neuroscience.

[14]  William B. Levy,et al.  Energy Efficient Neural Codes , 1996, Neural Computation.

[15]  Zhuo Wang,et al.  Efficient Neural Codes That Minimize Lp Reconstruction Error , 2016, Neural Computation.

[16]  Xue-Xin Wei,et al.  Mutual Information, Fisher Information, and Efficient Coding , 2016, Neural Computation.

[17]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[18]  Peter H. Schiller,et al.  The ON and OFF channels of the visual system , 1992, Trends in Neurosciences.

[19]  J. A. Movshon,et al.  The dependence of response amplitude and variance of cat visual cortical neurones on stimulus contrast , 1981, Experimental Brain Research.

[20]  F. Attneave Some informational aspects of visual perception. , 1954, Psychological review.

[21]  Michael J. Berry,et al.  Metabolically Efficient Information Processing , 2001, Neural Computation.

[22]  S. Laughlin,et al.  An Energy Budget for Signaling in the Grey Matter of the Brain , 2001, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[23]  G. J. Tomko,et al.  Neuronal variability: non-stationary responses to identical visual stimuli. , 1974, Brain research.

[24]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[25]  Don H. Johnson,et al.  Optimal Stimulus Coding by Neural Populations Using Rate Codes , 2004, Journal of Computational Neuroscience.

[26]  Xue-Xin Wei,et al.  A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts , 2015, Nature Neuroscience.

[27]  S. Laughlin A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[28]  Tatyana O Sharpee,et al.  Critical and maximally informative encoding between neural populations in the retina , 2014, Proceedings of the National Academy of Sciences.

[29]  Nicolas Brunel,et al.  Mutual Information, Fisher Information, and Population Coding , 1998, Neural Computation.

[30]  M. McDonnell,et al.  Maximally informative stimuli and tuning curves for sigmoidal rate-coding neurons and populations. , 2008, Physical review letters.

[31]  Eero P. Simoncelli,et al.  Efficient coding of natural images with a population of noisy Linear-Nonlinear neurons , 2011, NIPS.

[32]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[33]  D. Macleod,et al.  Optimal nonlinear codes for the perception of natural colours , 2001, Network.

[34]  Rob R. de Ruyter van Steveninck,et al.  The metabolic cost of neural information , 1998, Nature Neuroscience.

[35]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[36]  Allan Gottschalk,et al.  Derivation of the Visual Contrast Response Function by Maximizing Information Rate , 2002, Neural Computation.

[37]  Eero P. Simoncelli,et al.  Efficient Sensory Encoding and Bayesian Inference with Heterogeneous Neural Populations , 2014, Neural Computation.

[38]  Matthias Bethge,et al.  Optimal Short-Term Population Coding: When Fisher Information Fails , 2002, Neural Computation.

[39]  Charles P. Ratliff,et al.  Retina is structured to process an excess of darkness in natural scenes , 2010, Proceedings of the National Academy of Sciences.