Could information theory provide an ecological theory of sensory processing?

The sensory pathways of animals are well adapted to processing a special class of signals, namely stimuli from the animal’s environment. An important fact about natural stimuli is that they are typically very redundant and hence the sampled representation of these signals formed by the array of sensory cells is inefficient. One could argue for some animals and pathways, as we do in this review, that efficiency of information representation in the nervous system has several evolutionary advantages. Consequently, one might expect that much of the processing in the early levels of these sensory pathways could be dedicated towards recoding incoming signals into a more efficient form. In this review, we explore the principle of efficiency of information representation as a design principle for sensory processing. We give a preliminary discussion on how this principle could be applied in general to predict neural processing and then discuss concretely some neural systems where it recently has been shown to be successful. In particular, we examine the fly’s LMC coding strategy and the mammalian retinal coding in the spatial, temporal and chromatic domains.

[1]  Fletcher Pratt,et al.  Secret and Urgent. , 1939 .

[2]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[3]  H JACOBSON,et al.  The informational capacity of the human eye. , 1950, Science.

[4]  Claude E. Shannon,et al.  Prediction and Entropy of Printed English , 1951 .

[5]  C. Harrison Experiments with linear prediction in television , 1952 .

[6]  E. Kretzmer Statistics of television signals , 1952 .

[7]  F. Attneave Some informational aspects of visual perception. , 1954, Psychological review.

[8]  G. A. Barnard,et al.  Statistical calculation of word entropies for four Western languages , 1955, IRE Trans. Inf. Theory.

[9]  William F. Schreiber,et al.  The measurement of third order probability distributions of television signals , 1956, IRE Trans. Inf. Theory.

[10]  George C. Sziklai,et al.  Some studies in the speed of visual perception , 1956, Electrical Engineering.

[11]  M. C. GOODALL,et al.  Performance of a Stochastic Net , 1960, Nature.

[12]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[13]  C. Enroth-Cugell,et al.  The contrast sensitivity of retinal ganglion cells of the cat , 1966, The Journal of physiology.

[14]  F. Campbell,et al.  Optical quality of the human eye , 1966, The Journal of physiology.

[15]  M. A. Bouman,et al.  Spatial Modulation Transfer in the Human Eye , 1967 .

[16]  N. Daw Colour‐coded ganglion cells in the goldfish retina: extension of their receptive fields by means of new stimuli , 1968, The Journal of physiology.

[17]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[18]  D. H. Kelly Adaptation effects on spatio-temporal sine-wave thresholds. , 1972, Vision research.

[19]  H. Kornhuber Neural Control of Input into Long Term Memory: Limbic System and Amnestic Syndrome in Man , 1973 .

[20]  H. Davson Physiology of the Eye , 1951 .

[21]  R. L. de Valois,et al.  Psychophysical studies of monkey vision. 3. Spatial luminance contrast sensitivity tests of macaque and human observers. , 1974, Vision research.

[22]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[23]  J. Lythgoe The Ecology of vision , 1979 .

[24]  A. M. Uttley,et al.  Information transmission in the nervous system , 1979 .

[25]  S. Laughlin A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[26]  Satosi Watanabe,et al.  Pattern recognition as a quest for minimum entropy , 1981, Pattern Recognit..

[27]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[28]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[29]  G. Buchsbaum,et al.  Trichromacy, opponent colours coding and optimum colour information transmission in the retina , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[30]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  P. Lennie,et al.  Chromatic mechanisms in lateral geniculate nucleus of macaque. , 1984, The Journal of physiology.

[32]  S. Shaw Early visual processing in insects. , 1984, The Journal of experimental biology.

[33]  C. Enroth-Cugell,et al.  Chapter 9 Visual adaptation and retinal gain controls , 1984 .

[34]  S. Schein,et al.  Density profile of blue-sensitive cones along the horizontal meridian of macaque retina. , 1985, Investigative ophthalmology & visual science.

[35]  Satosi Watanabe,et al.  Pattern Recognition: Human and Mechanical , 1985 .

[36]  K. Mullen The contrast sensitivity of human colour vision to red‐green and blue‐yellow chromatic gratings. , 1985, The Journal of physiology.

[37]  Barak A. Pearlmutter,et al.  G-maximization: An unsupervised learning procedure for discovering regularities , 1987 .

[38]  H. Barlow,et al.  Human contrast discrimination and the threshold of cortical neurons. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[39]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[40]  Simon B. Laughlin,et al.  Form and function in retinal processing , 1987, Trends in Neurosciences.

[41]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[42]  Ralph Linsker,et al.  An Application of the Principle of Maximum Information Preservation to Linear Systems , 1988, NIPS.

[43]  David J. Field,et al.  What The Statistics Of Natural Images Tell Us About Visual Coding , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[44]  H. B. Barlow,et al.  Finding Minimum Entropy Codes , 1989, Neural Computation.

[45]  Simon B. Laughlin,et al.  Coding Efficiency and Design in Visual Processing , 1989 .

[46]  Richard Durbin,et al.  The computing neuron , 1989 .

[47]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[48]  Ralph Linsker,et al.  How to Generate Ordered Maps by Maximizing the Mutual Information between Input and Output Signals , 1989, Neural Computation.

[49]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[50]  Li Zhaoping,et al.  Color coding and its interaction with spatiotemporal processing in the retina , 1990 .

[51]  W. Bialek,et al.  Optimal Sampling of Natural Images: A Design Principle for the Visual System , 1990, NIPS 1990.

[52]  David C. Van Essen,et al.  Information processing strategies and pathways in the primate retina and visual cortex , 1990 .

[53]  Colin Blakemore,et al.  Statistical limits to image understanding , 1991 .

[54]  H. Barlow,et al.  Minimum-entropy coding with Hopfield networks , 1991 .

[55]  Bruno A. Olshausen,et al.  Pattern recognition, attention, and information bottlenecks in the primate visual system , 1991, Defense, Security, and Sensing.

[56]  Zhaoping Li,et al.  Understanding Retinal Color Coding from First Principles , 1992, Neural Computation.

[57]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[58]  W. Bialek,et al.  Reading Between the Spikes in the Cereal Filiform Hair Receptors of the Cricket , 1992 .

[59]  Zhaoping Li,et al.  What does post-adaptation color appearance reveal about cortical color representation? , 1993, Vision Research.

[60]  Joseph J. Atick,et al.  Convergent Algorithm for Sensory Receptive Field Development , 1993, Neural Computation.

[61]  A. Norman Redlich,et al.  Redundancy Reduction as a Strategy for Unsupervised Learning , 1993, Neural Computation.

[62]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[63]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .