Modeling Inhibitory Interneurons in Efficient Sensory Coding Models

There is still much unknown regarding the computational role of inhibitory cells in the sensory cortex. While modeling studies could potentially shed light on the critical role played by inhibition in cortical computation, there is a gap between the simplicity of many models of sensory coding and the biological complexity of the inhibitory subpopulation. In particular, many models do not respect that inhibition must be implemented in a separate subpopulation, with those inhibitory interneurons having a diversity of tuning properties and characteristic E/I cell ratios. In this study we demonstrate a computational framework for implementing inhibition in dynamical systems models that better respects these biophysical observations about inhibitory interneurons. The main approach leverages recent work related to decomposing matrices into low-rank and sparse components via convex optimization, and explicitly exploits the fact that models and input statistics often have low-dimensional structure that can be exploited for efficient implementations. While this approach is applicable to a wide range of sensory coding models (including a family of models based on Bayesian inference in a linear generative model), for concreteness we demonstrate the approach on a network implementing sparse coding. We show that the resulting implementation stays faithful to the original coding goals while using inhibitory interneurons that are much more biophysically plausible.

[1]  S. Laughlin,et al.  Energy limitation as a selective pressure on the evolution of sensory systems , 2008, Journal of Experimental Biology.

[2]  Christian K. Machens,et al.  Predictive Coding of Dynamical Variables in Balanced Spiking Networks , 2013, PLoS Comput. Biol..

[3]  Christopher J. Rozell,et al.  Visual Nonclassical Receptive Field Effects Emerge from Sparse Coding in a Dynamical System , 2013, PLoS Comput. Biol..

[4]  Christopher J. Rozell,et al.  Low Power Sparse Approximation on Reconfigurable Analog Hardware , 2012, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[5]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[6]  Tao Hu,et al.  A Network of Spiking Neurons for Computing Sparse Representations in an Energy-Efficient Way , 2012, Neural Computation.

[7]  Li I. Zhang,et al.  Visual Representations by Cortical Somatostatin Inhibitory Neurons—Selective But with Weak and Delayed Responses , 2010, The Journal of Neuroscience.

[8]  L. Martinez,et al.  Circuits that build visual cortical receptive fields , 2006, Trends in Neurosciences.

[9]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[10]  M. Scanziani,et al.  How Inhibition Shapes Cortical Activity , 2011, Neuron.

[11]  Christopher J. Rozell,et al.  Short-Term Memory Capacity in Networks via the Restricted Isometry Property , 2013, Neural Computation.

[12]  Eero P. Simoncelli Vision and the statistics of the visual environment , 2003, Current Opinion in Neurobiology.

[13]  M. Scanziani,et al.  Inhibition of Inhibition in Visual Cortex: The Logic of Connections Between Molecularly Distinct Interneurons , 2013, Nature Neuroscience.

[14]  Gilad Silberberg,et al.  Polysynaptic subcircuits in the neocortex: spatial and temporal diversity , 2008, Current Opinion in Neurobiology.

[15]  Christopher J. Rozell,et al.  A Common Network Architecture Efficiently Implements a Variety of Sparsity-Based Inference Problems , 2012, Neural Computation.

[16]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[17]  Arno C. Schmitt,et al.  Inhibitory interneurons in a cortical column form hot zones of inhibition in layers 2 and 5A , 2011, Proceedings of the National Academy of Sciences.

[18]  Nathan R. Wilson,et al.  Response Features of Parvalbumin-Expressing Interneurons Suggest Precise Roles for Subtypes of Inhibition in Visual Cortex , 2010, Neuron.

[19]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[20]  Cynthia Sámano,et al.  Neurotransmitter segregation: Functional and plastic implications , 2012, Progress in Neurobiology.

[21]  Christopher J. Rozell,et al.  Configurable hardware integrate and fire neurons for sparse approximation , 2013, Neural Networks.

[22]  N. Stanietsky,et al.  The interaction of TIGIT with PVR and PVRL2 inhibits human NK cell cytotoxicity , 2009, Proceedings of the National Academy of Sciences.

[23]  Richard G. Baraniuk,et al.  Sparse Coding via Thresholding and Local Competition in Neural Circuits , 2008, Neural Computation.

[24]  Matthew R. Krause,et al.  Synaptic and Network Mechanisms of Sparse and Reliable Visual Cortical Activity during Nonclassical Receptive Field Stimulation , 2010, Neuron.

[25]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[26]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[27]  M. Larkum A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex , 2013, Trends in Neurosciences.

[28]  P. J. Sjöström,et al.  Functional specificity of local synaptic connections in neocortical networks , 2011, Nature.

[29]  Rajesh P. N. Rao,et al.  Probabilistic Models of the Brain: Perception and Neural Function , 2002 .

[30]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[31]  Justin K. Romberg,et al.  Convergence and Rate Analysis of Neural Networks for Sparse Approximation , 2011, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Justin K. Romberg,et al.  Convergence of a neural network for sparse approximation using the nonsmooth Łojasiewicz inequality , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[33]  Kim Steenstrup Pedersen,et al.  The Nonlinear Statistics of High-Contrast Patches in Natural Images , 2003, International Journal of Computer Vision.

[34]  Martin Rehn,et al.  A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields , 2007, Journal of Computational Neuroscience.

[35]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[36]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[37]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[38]  A. Borst Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[39]  Joel Zylberberg,et al.  Inhibitory Interneurons Decorrelate Excitatory Cells to Drive Sparse Code Formation in a Spiking Model of V1 , 2013, The Journal of Neuroscience.

[40]  Nathan R. Wilson,et al.  Division and subtraction by distinct cortical inhibitory networks in vivo , 2012, Nature.

[41]  Arthur W. Wetzel,et al.  Network anatomy and in vivo physiology of visual cortical neurons , 2011, Nature.

[42]  Aditya Joshi,et al.  Cleaning up toxic waste: Removing nefarious contributions to recommendation systems , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[43]  U. Ernst,et al.  Perceptual Inference Predicts Contextual Modulations of Sensory Responses , 2012, The Journal of Neuroscience.

[44]  Gábor Szabó,et al.  Cannabinoid sensitivity and synaptic properties of 2 GABAergic networks in the neocortex. , 2008, Cerebral cortex.

[45]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[46]  E. B. Baum,et al.  Internal representations for associative memory , 1988, Biological Cybernetics.

[47]  H. Adesnik,et al.  A neural circuit for spatial summation in visual cortex , 2012, Nature.

[48]  Christopher J. Rozell,et al.  Optimal Sparse Approximation with Integrate and Fire Neurons , 2014, Int. J. Neural Syst..

[49]  Kechen Zhang,et al.  How to Modify a Neural Network Gradually Without Changing Its Input-Output Functionality , 2010, Neural Computation.

[50]  Maria V. Sanchez-Vives,et al.  Lack of orientation and direction selectivity in a subgroup of fast-spiking inhibitory interneurons: cellular and synaptic mechanisms and comparison with other electrophysiological cell types. , 2008, Cerebral cortex.

[51]  Michael Robert DeWeese,et al.  A Sparse Coding Model with Synaptically Local Plasticity and Spiking Neurons Can Account for the Diverse Shapes of V1 Simple Cell Receptive Fields , 2011, PLoS Comput. Biol..

[52]  W. Maass,et al.  State-dependent computations: spatiotemporal processing in cortical networks , 2009, Nature Reviews Neuroscience.

[53]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[54]  M. Brecht,et al.  Sparse and powerful cortical spikes , 2010, Current Opinion in Neurobiology.

[55]  R. Yuste,et al.  Dense, Unspecific Connectivity of Neocortical Parvalbumin-Positive Interneurons: A Canonical Microcircuit for Inhibition? , 2011, The Journal of Neuroscience.

[56]  Nikos K Logothetis,et al.  Statistical comparison of spike responses to natural stimuli in monkey area V1 with simulated responses of a detailed laminar network model for a patch of V1. , 2011, Journal of neurophysiology.

[57]  Karl Deisseroth,et al.  Activation of Specific Interneurons Improves V1 Feature Selectivity and Visual Perception , 2012, Nature.

[58]  E. Callaway,et al.  Excitatory cortical neurons form fine-scale functional networks , 2005, Nature.

[59]  Terrence J Sejnowski,et al.  Communication in Neuronal Networks , 2003, Science.

[60]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[61]  M. A. Repucci,et al.  Spatial Structure and Symmetry of Simple-Cell Receptive Fields in Macaque Primary Visual Cortex , 2002 .

[62]  Hongkui Zeng,et al.  Differential tuning and population dynamics of excitatory and inhibitory neurons reflect differences in local intracortical connectivity , 2011, Nature Neuroscience.

[63]  David J. Field,et al.  How Close Are We to Understanding V1? , 2005, Neural Computation.

[64]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[65]  C. Niell,et al.  What can mice tell us about how vision works? , 2011, Trends in Neurosciences.

[66]  H. S. Meyer,et al.  Cellular organization of cortical barrel columns is whisker-specific , 2013, Proceedings of the National Academy of Sciences.

[67]  A. Koulakov,et al.  Sparse Incomplete Representations: A Potential Role of Olfactory Granule Cells , 2011, Neuron.

[68]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[69]  P. Strata,et al.  Dale’s principle , 1999, Brain Research Bulletin.

[70]  Jose-Manuel Alonso,et al.  Functionally distinct inhibitory neurons at the first stage of visual cortical processing , 2003, Nature Neuroscience.

[71]  R. Reid,et al.  Specificity and randomness: structure–function relationships in neural circuits , 2011, Current Opinion in Neurobiology.

[72]  D. Ferster,et al.  Neural mechanisms of orientation selectivity in the visual cortex. , 2000, Annual review of neuroscience.

[73]  H. Markram,et al.  Interneurons of the neocortical inhibitory system , 2004, Nature Reviews Neuroscience.

[74]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.