Maximum Entropy Principle Analysis in Network Systems with Short-time Recordings

In many realistic systems, maximum entropy principle (MEP) analysis provides an effective characterization of the probability distribution of network states. However, to implement the MEP analysis, a sufficiently long-time data recording in general is often required, e.g., hours of spiking recordings of neurons in neuronal networks. The issue of whether the MEP analysis can be successfully applied to network systems with data from short-time recordings has yet to be fully addressed. In this work, we investigate relationships underlying the probability distributions, moments, and effective interactions in the MEP analysis and then show that, with short-time recordings of network dynamics, the MEP analysis can be applied to reconstructing probability distributions of network states that is much more accurate than the one directly measured from the short-time recording. Using spike trains obtained from both Hodgkin-Huxley neuronal networks and electrophysiological experiments, we verify our results and demonstrate that MEP analysis provides a tool to investigate the neuronal population coding properties for short-time recordings.

[1]  M. Bethge,et al.  Common input explains higher-order correlations and entropy in a simple model of neural population activity. , 2011, Physical review letters.

[2]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[3]  Front , 2020, 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4).

[4]  Robert E. Schapire,et al.  Faster solutions of the inverse pairwise Ising problem , 2008 .

[5]  Ifije E. Ohiorhenuan,et al.  Sparse coding and high-order correlations in fine-scale cortical networks , 2010, Nature.

[6]  D. Garlaschelli,et al.  Early-warning signals of topological collapse in interbank networks , 2013, Scientific Reports.

[7]  Michael J. Berry,et al.  Weak pairwise correlations imply strongly correlated network states in a neural population , 2005, Nature.

[8]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[9]  John Hertz,et al.  Ising Models for Inferring Network Structure From Spike Data , 2011, 1106.1752.

[10]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[11]  S. Khatiwala,et al.  Reconstruction of the history of anthropogenic CO2 concentrations in the ocean , 2009, Nature.

[12]  John M. Beggs,et al.  A Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical Networks In Vitro , 2008, The Journal of Neuroscience.

[13]  Gerhard Hummer,et al.  A maximum entropy approach to the study of residue‐specific backbone angle distributions in α‐synuclein, an intrinsically disordered protein , 2014, Protein science : a publication of the Protein Society.

[14]  Zhi-Qin John Xu,et al.  Granger Causality Network Reconstruction of Conductance-Based Integrate-and-Fire Neuronal Systems , 2014, PloS one.

[15]  A. Plastino,et al.  The workings of the maximum entropy principle in collective human behaviour , 2012, Journal of The Royal Society Interface.

[16]  S. Cocco,et al.  Ising models for neural activity inferred via selective cluster expansion: structural and coding properties , 2013 .

[17]  Bruno Cessac,et al.  Spatio-temporal spike train analysis for large scale networks using the maximum entropy principle and Monte Carlo method , 2012, 1209.3886.

[18]  Gasper Tkacik,et al.  Probabilistic models for neural populations that naturally capture global coupling and criticality , 2017, PLoS Comput. Biol..

[19]  Eric Shea-Brown,et al.  When do microcircuits produce beyond-pairwise correlations? , 2014, Front. Comput. Neurosci..

[20]  Jonathon Shlens,et al.  Synchronized firing in the retina , 2008, Current Opinion in Neurobiology.

[21]  J. Hertz,et al.  Ising model for neural data: model quality and approximate methods for extracting functional connectivity. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Bruno Cessac,et al.  Parameter Estimation for Spatio-Temporal Maximum Entropy Distributions: Application to Neural Spike Trains , 2014, Entropy.

[23]  Shan Yu,et al.  A Small World of Neuronal Synchrony , 2008, Cerebral cortex.

[24]  Douglas J. Miller,et al.  Maximum entropy econometrics: robust estimation with limited data , 1996 .

[25]  Yuji Ikegaya,et al.  Simultaneous silence organizes structured higher-order interactions in neural populations , 2015, Scientific Reports.

[26]  T. Bury.  Statistical pairwise interaction model of stock market , 2012, 1206.4420.

[27]  Ilya Nemenman,et al.  On the Sufficiency of Pairwise Interactions in Maximum Entropy Models of Networks , 2015, Journal of Statistical Physics.

[28]  Mattias P. Karlsson,et al.  Awake replay of remote experiences in the hippocampus , 2009, Nature Neuroscience.

[29]  Zhi-Qin John Xu,et al.  A dynamical state underlying the second order maximum entropy principle in neuronal networks , 2017 .

[30]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[31]  Ari S. Morcos,et al.  History-dependent variability in population dynamics during evidence accumulation in cortex , 2016, Nature Neuroscience.

[32]  Rémi Monasson,et al.  Learning probability distributions from smooth observables and the maximum entropy principle: Some remarks , 2015 .

[33]  Jiwei Zhang,et al.  A coarse-grained framework for spiking neuronal networks: between homogeneity and synchrony , 2014, Journal of Computational Neuroscience.

[34]  Shan Yu,et al.  Higher-Order Interactions Characterized in Cortical Activity , 2011, The Journal of Neuroscience.

[35]  F. Morcos,et al.  Genomics-aided structure prediction , 2012, Proceedings of the National Academy of Sciences.

[36]  Peter E. Latham,et al.  Pairwise Maximum Entropy Models for Studying Large Biological Systems: When They Can Work and When They Can't , 2008, PLoS Comput. Biol..

[37]  Michael J. Berry,et al.  Searching for Collective Behavior in a Large Network of Sensory Neurons , 2013, PLoS Comput. Biol..

[38]  Aonan Tang,et al.  Maximum Entropy Approaches to Living Neural Networks , 2010, Entropy.

[39]  Eric Shea-Brown,et al.  A Moment-Based Maximum Entropy Model for Fitting Higher-Order Interactions in Neural Data , 2018, bioRxiv.

[40]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[41]  Yasser Roudi,et al.  Correlations and Functional Connections in a Population of Grid Cells , 2014, PLoS Comput. Biol..

[42]  Jonathon Shlens,et al.  The Structure of Large-Scale Synchronized Firing in Primate Retina , 2009, The Journal of Neuroscience.

[43]  Yi Sun,et al.  Library-based numerical reduction of the Hodgkin–Huxley neuron for network simulation , 2009, Journal of Computational Neuroscience.

[44]  L. Saulis,et al.  Limit theorems for large deviations , 1991 .

[45]  Saeed Saremi,et al.  Hierarchical model of natural images and the origin of scale invariance , 2013, Proceedings of the National Academy of Sciences.

[46]  Sami El Boustani,et al.  Prediction of spatiotemporal patterns of neural activity from pairwise correlations. , 2009, Physical review letters.

[47]  Jonathon Shlens,et al.  The Structure of Multi-Neuron Firing Patterns in Primate Retina , 2006, The Journal of Neuroscience.

[48]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[49]  O. Schwartz,et al.  Flexible Gating of Contextual Influences in Natural Vision , 2015, Nature Neuroscience.

[50]  Jiwei Zhang,et al.  Distribution of correlated spiking events in a population-based approach for Integrate-and-Fire networks , 2014, Journal of Computational Neuroscience.

[51]  Naoki Masuda,et al.  A pairwise maximum entropy model accurately describes resting-state human brain networks , 2013, Nature Communications.

[52]  Thierry Mora,et al.  Dynamical maximum entropy approach to flocking. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[53]  R. Segev,et al.  Sparse low-order interaction network underlies a highly correlated and learnable neural population code , 2011, Proceedings of the National Academy of Sciences.

[54]  W. Bialek,et al.  Statistical mechanics of letters in words. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[55]  Masaki Mishima,et al.  Protein structure determination in living cells by in-cell NMR spectroscopy , 2009, Nature.

[56]  R. Segev,et al.  The Architecture of Functional Interaction Networks in the Retina , 2011, The Journal of Neuroscience.

[57]  Yi Sun,et al.  Pseudo-Lyapunov exponents and predictability of Hodgkin-Huxley neuronal network dynamics , 2010, Journal of Computational Neuroscience.