Nonlinear scaling of resource allocation in sensory bottlenecks

In many sensory systems, information transmission is constrained by a bottleneck, where the number of output neurons is vastly smaller than the number of input neurons. Efficient coding theory predicts that in these scenarios the brain should allocate its limited resources by removing redundant information. Previous work has typically assumed that receptors are uniformly distributed across the sensory sheet, when in reality these vary in density, often by an order of magnitude. How, then, should the brain efficiently allocate output neurons when the density of input neurons is nonuniform? Here, we show analytically and numerically that resource allocation scales nonlinearly in efficient coding models that maximize information transfer, when inputs arise from separate regions with different receptor densities. Importantly, the proportion of output neurons allocated to a given input region changes depending on the width of the bottleneck, and thus cannot be predicted from input density or region size alone. Narrow bottlenecks favor magnification of high density input regions, while wider bottlenecks often cause contraction. Our results demonstrate that both expansion and contraction of sensory input regions can arise in efficient coding models and that the final allocation crucially depends on the neural resources made available.

[1]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[2]  Michael S. Lewicki,et al.  A Simple Model of Optimal Population Coding for Sensory Systems , 2014, PLoS Comput. Biol..

[3]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[4]  Daniel J. Graham,et al.  Can the theory of “whitening” explain the center-surround properties of retinal ganglion cell receptive fields? , 2006, Vision Research.

[5]  Charles F Stevens,et al.  Predicting Functional Properties of Visual Cortex from an Evolutionary Scaling Law , 2002, Neuron.

[6]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[7]  Elaine M. Wells-Gray,et al.  Variation in rod and cone density from the fovea to the mid-periphery in healthy human retinas using adaptive optics scanning laser ophthalmoscopy , 2016, Eye.

[8]  Paul R. Martin,et al.  Comparison of photoreceptor spatial density and ganglion cell morphology in the retina of human, macaque monkey, cat, and the marmoset Callithrix jacchus , 1996, The Journal of comparative neurology.

[9]  R. Johansson,et al.  Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. , 1979, The Journal of physiology.

[10]  Joseph J Atick,et al.  Could information theory provide an ecological theory of sensory processing? , 2011, Network.

[11]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[12]  Eero P. Simoncelli,et al.  Efficient Sensory Encoding and Bayesian Inference with Heterogeneous Neural Populations , 2014, Neural Computation.

[13]  Qi Zhao,et al.  SALICON: Saliency in Context , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Surya Ganguli,et al.  A Unified Theory Of Early Visual Representations From Retina To Cortex Through Anatomically Constrained Deep CNNs , 2019, bioRxiv.

[15]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[16]  Eero P. Simoncelli,et al.  Efficient Coding of Spatial Information in the Primate Retina , 2012, The Journal of Neuroscience.

[17]  Xue-Xin Wei,et al.  A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts , 2015, Nature Neuroscience.

[18]  K. Strimmer,et al.  Optimal Whitening and Decorrelation , 2015, 1512.00809.

[19]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[20]  Simona Cocco,et al.  Adaptation of olfactory receptor abundances for efficient coding , 2019, eLife.

[21]  K. Veselic,et al.  Some remarks on the spectra of Hermitian matrices , 1991 .

[22]  F. Attneave Some informational aspects of visual perception. , 1954, Psychological review.