Dimension reduction in heterogeneous neural networks: Generalized Polynomial Chaos (gPC) and ANalysis-Of-VAriance (ANOVA)

Abstract We propose, and illustrate via a neural network example, two different approaches to coarse-graining large heterogeneous networks. Both approaches are inspired from, and use tools developed in, methods for uncertainty quantification (UQ) in systems with multiple uncertain parameters – in our case, the parameters are heterogeneously distributed on the network nodes. The approach shows promise in accelerating large scale network simulations as well as coarse-grained fixed point, periodic solution computation and stability analysis. We also demonstrate that the approach can successfully deal with structural as well as intrinsic heterogeneities.

[1]  Ioannis G. Kevrekidis,et al.  Periodically-forced finite networks of heterogeneous globally-coupled oscillators: A low-dimensional approach , 2008 .

[2]  I. Kevrekidis,et al.  Coarse graining the dynamics of coupled oscillators. , 2005, Physical review letters.

[3]  Jonathan E. Rubin,et al.  Optimal Intrinsic Dynamics for Bursting in a Three-Cell Network , 2010, SIAM J. Appl. Dyn. Syst..

[4]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[5]  Yu Zou,et al.  An equation-free approach to analyzing heterogeneous cell population dynamics , 2007, Journal of mathematical biology.

[6]  Dongbin Xiu,et al.  High-Order Collocation Methods for Differential Equations with Random Inputs , 2005, SIAM J. Sci. Comput..

[7]  I. Sobol Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[8]  J. C. Smith,et al.  Models of respiratory rhythm generation in the pre-Bötzinger complex. I. Bursting pacemaker neurons. , 1999, Journal of neurophysiology.

[9]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[10]  J. Herskowitz,et al.  Proceedings of the National Academy of Sciences, USA , 1996, Current Biology.

[11]  Carlo R Laing,et al.  Managing heterogeneity in the study of neural oscillator dynamics , 2012, The Journal of Mathematical Neuroscience.

[12]  Sergej O. Voronenko,et al.  Shifting Spike Times or Adding and Deleting Spikes—How Different Types of Noise Shape Signal Transmission in Neural Populations , 2015, Journal of mathematical neuroscience.

[13]  I. Kevrekidis,et al.  "Coarse" stability and bifurcation analysis using time-steppers: a reaction-diffusion example. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Carlo R Laing,et al.  Coarse-Grained Clustering Dynamics of Heterogeneously Coupled Neurons , 2015, Journal of mathematical neuroscience.

[15]  B. Hassard Bifurcation of periodic solutions of Hodgkin-Huxley model for the squid giant axon. , 1978, Journal of theoretical biology.

[16]  M. Kendall Statistical Methods for Research Workers , 1937, Nature.

[17]  Thomas Gerstner,et al.  Numerical integration using sparse grids , 2004, Numerical Algorithms.

[18]  Carlo R Laing,et al.  Chimeras in random non-complete networks of phase oscillators. , 2012, Chaos.

[19]  George E. Karniadakis,et al.  The multi-element probabilistic collocation method (ME-PCM): Error analysis and applications , 2008, J. Comput. Phys..

[20]  C. W. Gear,et al.  Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis , 2003 .

[21]  P. Ashwin,et al.  The dynamics ofn weakly coupled identical oscillators , 1992 .

[22]  Jasmine Yen-teng Foo,et al.  Multi-element probabilistic collocation in high dimensions: Applications to systems biology and physical systems , 2008 .

[23]  J. C. Smith,et al.  Models of respiratory rhythm generation in the pre-Bötzinger complex. II. Populations Of coupled pacemaker neurons. , 1999, Journal of neurophysiology.

[24]  W. Hoeffding A Class of Statistics with Asymptotically Normal Distribution , 1948 .

[25]  George E. Karniadakis,et al.  Beyond Wiener–Askey Expansions: Handling Arbitrary PDFs , 2006, J. Sci. Comput..

[26]  George E. Karniadakis,et al.  Multi-element probabilistic collocation method in high dimensions , 2010, J. Comput. Phys..

[27]  Ioannis G. Kevrekidis,et al.  Projective Methods for Stiff Differential Equations: Problems with Gaps in Their Eigenvalue Spectrum , 2002, SIAM J. Sci. Comput..

[28]  Jonathan E Rubin,et al.  Bursting induced by excitatory synaptic coupling in nonidentical conditional relaxation oscillators or square-wave bursters. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  Jonathan E. Rubin,et al.  Synchronized Activity and Loss of Synchrony Among Heterogeneous Conditional Oscillators , 2002, SIAM J. Appl. Dyn. Syst..

[30]  Xiu Yang,et al.  Adaptive ANOVA decomposition of stochastic incompressible and compressible flows , 2012, J. Comput. Phys..

[31]  D. Xiu Numerical Methods for Stochastic Computations: A Spectral Method Approach , 2010 .