DNN-assisted statistical analysis of a model of local cortical circuits

In neuroscience, computational modeling is an effective way to gain insight into cortical mechanisms, yet the construction and analysis of large-scale network models—not to mention the extraction of underlying principles—are themselves challenging tasks, due to the absence of suitable analytical tools and the prohibitive costs of systematic numerical exploration of high-dimensional parameter spaces. In this paper, we propose a data-driven approach assisted by deep neural networks (DNN). The idea is to first discover certain input-output relations, and then to leverage this information and the superior computation speeds of the well-trained DNN to guide parameter searches and to deduce theoretical understanding. To illustrate this novel approach, we used as a test case a medium-size network of integrate-and-fire neurons intended to model local cortical circuits. With the help of an accurate yet extremely efficient DNN surrogate, we revealed the statistics of model responses, providing a detailed picture of model behavior. The information obtained is both general and of a fundamental nature, with direct application to neuroscience. Our results suggest that the methodology proposed can be scaled up to larger and more complex biological networks when used in conjunction with other techniques of biological modeling.

[1]  R. Shapley,et al.  Rhythm and Synchrony in a Cortical Network Model , 2018, The Journal of Neuroscience.

[2]  Atharv Bhosekar,et al.  Advances in surrogate based modeling, feasibility analysis, and optimization: A review , 2018, Comput. Chem. Eng..

[3]  Lei Wu,et al.  Machine Learning from a Continuous Viewpoint , 2019, ArXiv.

[4]  D. R. Muir,et al.  Assessing the Role of Inhibition in Stabilizing Neocortical Networks Requires Large-Scale Perturbation of the Inhibitory Population , 2017, The Journal of Neuroscience.

[5]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[6]  Binghuang Cai,et al.  Systematic Integration of Structural and Functional Data into Multi-scale Models of Mouse Primary Visual Cortex , 2019, Neuron.

[7]  Lai-Sang Young,et al.  Orientation Selectivity from Very Sparse LGN Inputs in a Comprehensive Model of Macaque V1 Cortex , 2016, The Journal of Neuroscience.

[8]  Yoshua Bengio,et al.  On the Spectral Bias of Deep Neural Networks , 2018, ArXiv.

[9]  Bryan A. Tolson,et al.  Review of surrogate modeling in water resources , 2012 .

[10]  David Hansel,et al.  Mechanisms underlying the response of mouse cortical networks to optogenetic manipulation , 2019, bioRxiv.

[11]  Jessica A. Cardin,et al.  Stimulus Feature Selectivity in Excitatory and Inhibitory Neurons in Primary Visual Cortex , 2007, The Journal of Neuroscience.

[12]  Haim Sompolinsky,et al.  Chaotic Balanced State in a Model of Cortical Circuits , 1998, Neural Computation.

[13]  R. Shapley,et al.  A neuronal network model of macaque primary visual cortex (V1): orientation selectivity and dynamics in the input layer 4Calpha. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[14]  David Hansel,et al.  Asynchronous Rate Chaos in Spiking Neuronal Circuits , 2015, bioRxiv.

[15]  D. Lewis,et al.  Alterations of Cortical GABA Neurons and Network Oscillations in Schizophrenia , 2010, Current psychiatry reports.

[16]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[17]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[18]  ShiNung Ching,et al.  Dynamical changes in neurological diseases and anesthesia , 2012, Current Opinion in Neurobiology.

[19]  Daniel B. Rubin,et al.  The Stabilized Supralinear Network: A Unifying Circuit Motif Underlying Multi-Input Integration in Sensory Cortex , 2015, Neuron.

[20]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[21]  Gang Sun,et al.  A review of the artificial neural network surrogate modeling in aerodynamic design , 2019, Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering.

[22]  Zhi-Qin John Xu,et al.  Training behavior of deep neural network in frequency domain , 2018, ICONIP.

[23]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[24]  B. McNaughton,et al.  Paradoxical Effects of External Modulation of Inhibitory Interneurons , 1997, The Journal of Neuroscience.

[25]  Moritz Helias,et al.  Fundamental Activity Constraints Lead to Specific Interpretations of the Connectome , 2015, PLoS Comput. Biol..

[26]  E Weinan,et al.  On the Generalization Properties of Minimum-norm Solutions for Over-parameterized Neural Network Models , 2019, ArXiv.

[27]  Zheng Ma,et al.  Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks , 2019, Communications in Computational Physics.

[28]  Markus Diesmann,et al.  A multi-scale layer-resolved spiking network model of resting-state dynamics in macaque visual cortical areas , 2018, PLoS Comput. Biol..

[29]  M. Wehr,et al.  Rapid Rebalancing of Excitation and Inhibition by Cortical Circuitry , 2018, Neuron.

[30]  Linfeng Zhang,et al.  DeePCG: Constructing coarse-grained models via deep neural networks. , 2018, The Journal of chemical physics.

[31]  Naftali Tishby,et al.  Machine learning and the physical sciences , 2019, Reviews of Modern Physics.

[32]  Brent Doiron,et al.  Inhibitory stabilization and visual coding in cortical circuits with multiple interneuron subtypes. , 2016, Journal of neurophysiology.

[33]  R. Shapley,et al.  Contrast response in a comprehensive network model of macaque V1 , 2020, Journal of vision.

[34]  R. Shapley,et al.  LFP power spectra in V1 cortex: the graded effect of stimulus contrast. , 2005, Journal of neurophysiology.

[35]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Arvind Kumar,et al.  Homologous Basal Ganglia Network Models in Physiological and Parkinsonian Conditions , 2017, Front. Comput. Neurosci..

[38]  E Weinan,et al.  DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamics , 2017, Comput. Phys. Commun..

[39]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[40]  W. Andrew Pruett,et al.  The Creation of Surrogate Models for Fast Estimation of Complex Model Outcomes , 2016, PloS one.

[41]  James G. King,et al.  Reconstruction and Simulation of Neocortical Microcircuitry , 2015, Cell.

[42]  Lai Sang Young,et al.  Towards a Mathematical Model of the Brain , 2020, Journal of Statistical Physics.

[43]  Evan S. Schaffer,et al.  Inhibitory Stabilization of the Cortical Network Underlies Visual Surround Suppression , 2009, Neuron.

[44]  Dongbin Xiu,et al.  Parameter uncertainty quantification using surrogate models applied to a spatial model of yeast mating polarization , 2018, PLoS Comput. Biol..

[45]  J. Rinn,et al.  DeCoN: Genome-wide Analysis of In Vivo Transcriptional Dynamics during Pyramidal Neuron Fate Selection in Neocortex , 2015, Neuron.

[46]  H. Swadlow Efferent neurons and suspected interneurons in binocular visual cortex of the awake rabbit: receptive fields and binocular properties. , 1988, Journal of neurophysiology.

[47]  R. Shapley,et al.  Visual spatial characterization of macaque V1 neurons. , 2001, Journal of neurophysiology.

[48]  Jorge F Mejias,et al.  Paradoxical response reversal of top-down modulation in cortical circuits with three interneuron types. , 2017, eLife.

[49]  Marc W Howard,et al.  Theta and Gamma Oscillations during Encoding Predict Subsequent Recall , 2003, The Journal of Neuroscience.

[50]  Zheng Ma,et al.  Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks , 2019, ArXiv.

[51]  A. Angelucci,et al.  Circuits and Mechanisms for Surround Modulation in Visual Cortex. , 2017, Annual review of neuroscience.

[52]  Jeffry S. Isaacson,et al.  Network-Level Control of Frequency Tuning in Auditory Cortex , 2017, Neuron.

[53]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[54]  Lai-Sang Young,et al.  Emergent spike patterns in neuronal populations , 2014, Journal of Computational Neuroscience.

[55]  D. G. Albrecht,et al.  Striate cortex of monkey and cat: contrast response function. , 1982, Journal of neurophysiology.

[56]  L. Abbott,et al.  Neural network dynamics. , 2005, Annual review of neuroscience.