Statistical analysis and optimality of neural systems

Normative theories and statistical inference provide complementary approaches for the study of biological systems. A normative theory postulates that organisms have adapted to efficiently solve essential tasks, and proceeds to mathematically work out testable consequences of such optimality; parameters that maximize the hypothesized organismal function can be derived ab initio, without reference to experimental data. In contrast, statistical inference focuses on efficient utilization of data to learn model parameters, without reference to any a priori notion of biological function, utility, or fitness. Traditionally, these two approaches were developed independently and applied separately. Here we unify them in a coherent Bayesian framework that embeds a normative theory into a family of maximum-entropy “optimization priors.” This family defines a smooth interpolation between a data-rich inference regime (characteristic of “bottom-up” statistical models), and a data-limited ab inito prediction regime (characteristic of “top-down” normative theory). We demonstrate the applicability of our framework using data from the visual cortex, the retina, and C. elegans, and argue that the flexibility it affords is essential to address a number of fundamental challenges relating to inference and prediction in complex, high-dimensional biological problems.

[1]  Aljoscha Nern,et al.  Neural signatures of dynamic stimulus selection in Drosophila , 2017, Nature Neuroscience.

[2]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[3]  Steven C Dakin,et al.  An oblique effect for local motion: psychophysics and natural movie statistics. , 2005, Journal of vision.

[4]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[5]  Charles P. Ratliff,et al.  Design of a Neuronal Array , 2008, The Journal of Neuroscience.

[6]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[7]  Gonzalo G de Polavieja,et al.  Structure of deviations from optimality in biological systems , 2009, Proceedings of the National Academy of Sciences.

[8]  Wei-Chia Chen,et al.  Density estimation on small datasets , 2018, Physical review letters.

[9]  D. Ringach Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. , 2002, Journal of neurophysiology.

[10]  Eve Marder,et al.  Computational models in the age of large datasets , 2015, Current Opinion in Neurobiology.

[11]  H. Sompolinsky,et al.  Benefits of Pathway Splitting in Sensory Coding , 2014, The Journal of Neuroscience.

[12]  R. Rosen Optimality Principles in Biology , 1967, Springer US.

[13]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[14]  Nicole L. Carlson,et al.  Sparse Codes for Speech Predict Spectrotemporal Receptive Fields in the Inferior Colliculus , 2012, PLoS Comput. Biol..

[15]  A. Pérez-Escudero,et al.  Optimally wired subnetwork determines neuroanatomy of Caenorhabditis elegans , 2007, Proceedings of the National Academy of Sciences.

[16]  Michael S. Lewicki,et al.  A Simple Model of Optimal Population Coding for Sensory Systems , 2014, PLoS Comput. Biol..

[17]  Michael Robert DeWeese,et al.  A Sparse Coding Model with Synaptically Local Plasticity and Spiking Neurons Can Account for the Diverse Shapes of V1 Simple Cell Receptive Fields , 2011, PLoS Comput. Biol..

[18]  William Bialek,et al.  Information processing in living systems , 2014, 1412.8752.

[19]  Matthias Bethge,et al.  Natural Image Coding in V1: How Much Use Is Orientation Selectivity? , 2008, PLoS Comput. Biol..

[20]  J. Atick,et al.  STATISTICS OF NATURAL TIME-VARYING IMAGES , 1995 .

[21]  Gašper Tkačik,et al.  Inferring the function performed by a recurrent neural network , 2019, PloS one.

[22]  S. Wright,et al.  THE DISTRIBUTION OF GENE FREQUENCIES IN POPULATIONS. , 1937, Science.

[23]  Michael S. Lewicki,et al.  Efficient auditory coding , 2006, Nature.

[24]  Dmitri B. Chklovskii,et al.  Exact Solution for the Optimal Neuronal Layout Problem , 2004, Neural Computation.

[25]  D. Ruderman,et al.  Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[26]  Tatyana O Sharpee,et al.  Computational identification of receptive fields. , 2013, Annual review of neuroscience.

[27]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[28]  Tatyana O Sharpee,et al.  Critical and maximally informative encoding between neural populations in the retina , 2014, Proceedings of the National Academy of Sciences.

[29]  Ann M Hermundstad,et al.  Adaptive coding for dynamic sensory inference , 2017, bioRxiv.

[30]  Gasper Tkacik,et al.  Optimal population coding by noisy spiking neurons , 2010, Proceedings of the National Academy of Sciences.

[31]  Laurenz Wiskott,et al.  Slow feature analysis yields a rich repertoire of complex cell properties. , 2005, Journal of vision.

[32]  Behram N. Kursunoglu,et al.  Information Processing in Biological Systems , 1985, Studies in the Natural Sciences.

[33]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[34]  H. P. de Vladar,et al.  Statistical Mechanics and the Evolution of Polygenic Quantitative Traits , 2009, Genetics.

[35]  Eero P. Simoncelli,et al.  Efficient Coding of Spatial Information in the Primate Retina , 2012, The Journal of Neuroscience.

[36]  Tsvi Tlusty,et al.  Cross-species analysis traces adaptation of Rubisco toward optimality in a low-dimensional landscape , 2010, Proceedings of the National Academy of Sciences.

[37]  W. Geisler,et al.  Contributions of ideal observer theory to vision research , 2011, Vision Research.

[38]  D. Landau,et al.  Efficient, multiple-range random walk algorithm to calculate the density of states. , 2000, Physical review letters.

[39]  M. Meister,et al.  Decorrelation and efficient coding by retinal ganglion cells , 2012, Nature Neuroscience.

[40]  Christopher R. Myers,et al.  Universally Sloppy Parameter Sensitivities in Systems Biology Models , 2007, PLoS Comput. Biol..

[41]  Eric Shea-Brown,et al.  How Do Efficient Coding Strategies Depend on Origins of Noise in Neural Circuits? , 2016, PLoS Comput. Biol..

[42]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[43]  Jakob H. Macke,et al.  Flexible statistical inference for mechanistic models of neural dynamics , 2017, NIPS.

[44]  O. Marre,et al.  Toward a unified theory of efficient, predictive, and sparse coding , 2017, Proceedings of the National Academy of Sciences.

[45]  Pierre Yger,et al.  Multiplexed computations in retinal ganglion cells of a single type , 2016, Nature Communications.

[46]  W. Bialek Biophysics: Searching for Principles , 2012 .

[47]  J. H. Hateren,et al.  Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation , 1992, Journal of Comparative Physiology A.

[48]  D. Chklovskii,et al.  Wiring optimization can relate neuronal structure and function. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[49]  B. Palsson,et al.  Escherichia coli K-12 undergoes adaptive evolution to achieve in silico predicted optimal growth , 2002, Nature.

[50]  A. Gelman Parameterization and Bayesian Modeling , 2004 .

[51]  A. Tero,et al.  Rules for Biologically Inspired Adaptive Network Design , 2010, Science.

[52]  Terrence J. Sejnowski,et al.  Spatiochromatic Receptive Field Properties Derived from Information-Theoretic Analyses of Cone Mosaic Responses to Natural Scenes , 2003, Neural Computation.

[53]  Wiktor Mlynarski,et al.  Learning Midlevel Auditory Codes from Natural Sound Statistics , 2017, Neural Computation.

[54]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[55]  S. Laughlin A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[56]  Il Memming Park,et al.  Bayesian Efficient Coding , 2017, bioRxiv.

[57]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[58]  Constant D. Beugré,et al.  The neural basis of decision making , 2018 .

[59]  G. Tkačik,et al.  Statistical mechanics for metabolic networks during steady state growth , 2017, Nature Communications.

[60]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[61]  Cristina Savin,et al.  Estimating Nonlinear Neural Response Functions using GP Priors and Kronecker Methods , 2016, NIPS.

[62]  Liam Paninski,et al.  Statistical models for neural encoding, decoding, and optimal stimulus design. , 2007, Progress in brain research.

[63]  Eero P. Simoncelli,et al.  Efficient Sensory Encoding and Bayesian Inference with Heterogeneous Neural Populations , 2014, Neural Computation.

[64]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[65]  W. Bialek,et al.  Neural Decision Boundaries for Maximal Information Transmission , 2007, PLoS ONE.

[66]  Charles P. Ratliff,et al.  Retina is structured to process an excess of darkness in natural scenes , 2010, Proceedings of the National Academy of Sciences.

[67]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[68]  D. M I T R I Exact solution for the optimal neuronal layout problem , 2003 .

[69]  H. Kacser,et al.  The control of flux. , 1995, Biochemical Society transactions.

[70]  Callan,et al.  Field Theories for Learning Probability Distributions. , 1996, Physical review letters.

[71]  H. Jeffreys An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[72]  Sean R. Bittner,et al.  Interrogating theoretical models of neural computation with deep inference , 2019, bioRxiv.

[73]  Zhuo Wang,et al.  Efficient Neural Codes That Minimize Lp Reconstruction Error , 2016, Neural Computation.

[74]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[75]  Mijung Park,et al.  Receptive Field Inference with Localized Priors , 2011, PLoS Comput. Biol..

[76]  Wiktor Mlynarski,et al.  The Opponent Channel Population Code of Sound Location Is an Efficient Representation of Natural Binaural Sounds , 2015, PLoS Comput. Biol..

[77]  T. Ohshima,et al.  Stimulated emission from nitrogen-vacancy centres in diamond , 2016, Nature Communications.

[78]  Emery N. Brown,et al.  Analysis of Neural Data , 2014 .

[79]  W. Bialek,et al.  Information flow and optimization in transcriptional regulation , 2007, Proceedings of the National Academy of Sciences.

[80]  R. McNeill Alexander,et al.  Principles of Animal Locomotion , 2002 .

[81]  J. Sethna,et al.  Parameter Space Compression Underlies Emergent Theories and Predictive Models , 2013, Science.