Estimating the dimensionality of the manifold underlying multi-electrode neural recordings

It is generally accepted that the number of neurons in a given brain area far exceeds the number of neurons needed to carry any specific function controlled by that area. For example, motor areas of the human brain contain tens of millions of neurons that control the activation of tens or at most hundreds of muscles. This massive redundancy implies the covariation of many neurons, which constrains the population activity to a low-dimensional manifold within the space of all possible patterns of neural activity. To gain a conceptual understanding of the complexity of the neural activity within a manifold, it is useful to estimate its dimensionality, which quantifies the number of degrees of freedom required to describe the observed population activity without significant information loss. While there are many algorithms for dimensionality estimation, we do not know which are well suited for analyzing neural activity. The objective of this study was to evaluate the efficacy of several representative algorithms for estimating the dimensionality of linearly and nonlinearly embedded data. We generated synthetic neural recordings with known intrinsic dimensionality and used them to test the algorithms’ accuracy and robustness. We emulated some of the important challenges associated with experimental data by adding noise, altering the nature of the embedding of the low-dimensional manifold within the high-dimensional recordings, varying the dimensionality of the manifold, and limiting the amount of available data. We demonstrated that linear algorithms overestimate the dimensionality of nonlinear, noise-free data. In cases of high noise, most algorithms overestimated the dimensionality. We thus developed a denoising algorithm based on deep learning, the “Joint Autoencoder”, which significantly improved subsequent dimensionality estimation. Critically, we found that all algorithms failed when the intrinsic dimensionality was high (above 20) or when the amount of data used for estimation was low. Based on the challenges we observed, we formulated a pipeline for estimating the dimensionality of experimental neural data.

[1]  D. Ruelle,et al.  Fundamental limitations for estimating dimensions and Lyapunov exponents in dynamical systems , 1992 .

[2]  Giovanni Bussi,et al.  Predicting the Kinetics of RNA Oligonucleotides Using Markov State Models. , 2016, Journal of chemical theory and computation.

[3]  P. Grassberger,et al.  Measuring the Strangeness of Strange Attractors , 1983 .

[4]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[5]  Byron M. Yu,et al.  Neural constraints on learning , 2014, Nature.

[6]  G. La Camera,et al.  Stimuli Reduce the Dimensionality of Cortical Activity , 2015, bioRxiv.

[7]  Konrad P Kording,et al.  How advances in neural recording affect data analysis , 2011, Nature Neuroscience.

[8]  A. Faisal,et al.  Noise in the nervous system , 2008, Nature Reviews Neuroscience.

[9]  Matemáticas Nonlinear Dimensionality Reduction , 2013 .

[10]  Anqi Wu,et al.  Gaussian process based nonlinear latent structure discovery in multivariate spike train data , 2017, NIPS.

[11]  Nicholas A. Steinmetz,et al.  High-dimensional geometry of population responses in visual cortex , 2018, Nature.

[12]  J. Horn A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[13]  Antonino Staiano,et al.  Intrinsic dimension estimation: Advances and open problems , 2016, Inf. Sci..

[14]  Mei Ying Boon,et al.  The correlation dimension: a useful objective measure of the transient visual evoked potential? , 2008, Journal of vision.

[15]  Surya Ganguli,et al.  A theory of multineuronal dimensionality, dynamics and measurement , 2017, bioRxiv.

[16]  Jochen Einbeck,et al.  Intrinsic Dimensionality Estimation for High-dimensional Data Sets: New Approaches for the Computation of Correlation Dimension , 2013 .

[17]  Krishna V. Shenoy,et al.  Accurate Estimation of Neural Population Dynamics without Spike Sorting , 2019, Neuron.

[18]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[19]  Eva L. Dyer,et al.  Latent Factors and Dynamics in Motor Cortex and Their Application to Brain–Machine Interfaces , 2018, The Journal of Neuroscience.

[20]  John P. Cunningham,et al.  Methods for estimating neural firing rates, and their application to brain-machine interfaces , 2009, Neural Networks.

[21]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[22]  Chethan Pandarinath,et al.  Inferring single-trial neural population dynamics using sequential auto-encoders , 2017, Nature Methods.

[23]  Andrei Yu. Zinovyev,et al.  Estimating the effective dimension of large biological datasets using Fisher separability analysis , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[24]  Brent Doiron,et al.  Scaling Properties of Dimensionality Reduction for Neural Populations and Network Models , 2016, PLoS Comput. Biol..

[25]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[26]  Ivan Tyukin,et al.  High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality , 2020, Entropy.

[27]  Christian K. Machens,et al.  Behavioral / Systems / Cognitive Functional , But Not Anatomical , Separation of “ What ” and “ When ” in Prefrontal Cortex , 2009 .

[28]  Lee E. Miller,et al.  Long-term stability of cortical population dynamics underlying consistent behavior , 2019, Nature Neuroscience.

[29]  G. Laurent,et al.  Transient Dynamics versus Fixed Points in Odor Representations by Locust Antennal Lobe Projection Neurons , 2005, Neuron.

[30]  Surya Ganguli,et al.  Accurate Estimation of Neural Population Dynamics without Spike Sorting , 2017, Neuron.

[31]  E. Evarts Pyramidal tract activity associated with a conditioned hand movement in the monkey. , 1966, Journal of neurophysiology.

[32]  Nicholas A. Steinmetz,et al.  Spontaneous behaviors drive multidimensional, brainwide activity , 2019, Science.

[33]  David Sussillo,et al.  Making brain–machine interfaces robust to future neural variability , 2016, Nature communications.

[34]  Yoshua Bengio,et al.  Adversarial Domain Adaptation for Stable Brain-Machine Interfaces , 2018, ICLR.

[35]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[36]  Brent Doiron,et al.  Bridging large-scale neuronal recordings and large-scale network models using dimensionality reduction , 2018, Current Opinion in Neurobiology.

[37]  A. Vinciarelli,et al.  Estimating the Intrinsic Dimension of Data with a Fractal-Based Method , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  P. Grassberger,et al.  Measuring the Strangeness of Strange Attractors , 1983 .

[39]  Francis R. Willett,et al.  High performance communication by people with paralysis using an intracortical brain-computer interface , 2017, eLife.

[40]  L. Abbott,et al.  Two layers of neural variability , 2012, Nature Neuroscience.

[41]  Ivan Tyukin,et al.  Correction of AI systems by linear discriminants: Probabilistic foundations , 2018, Inf. Sci..

[42]  Adrienne L. Fairhall,et al.  Dimensionality reduction in neuroscience , 2016, Current Biology.

[43]  Surya Ganguli,et al.  On simplicity and complexity in the brave new world of large-scale neuroscience , 2015, Current Opinion in Neurobiology.

[44]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[45]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[46]  Leonard A. Smith Intrinsic limits on dimension calculations , 1988 .

[47]  Jochen Einbeck,et al.  On the computation of the correlation integral for fractal dimension estimation , 2012, 2012 International Conference on Statistics in Science, Business and Engineering (ICSSBE).

[48]  Alessandro Laio,et al.  Estimating the intrinsic dimension of datasets by a minimal neighborhood information , 2017, Scientific Reports.

[49]  Peter J. Bickel,et al.  Maximum Likelihood Estimation of Intrinsic Dimension , 2004, NIPS.

[50]  Balázs Kégl,et al.  Intrinsic Dimension Estimation Using Packing Numbers , 2002, NIPS.

[51]  Emil Wärnberg,et al.  Perturbing low dimensional activity manifolds in spiking neuronal networks , 2019, PLoS Comput. Biol..

[52]  P. Campadelli,et al.  Intrinsic Dimension Estimation: Relevant Techniques and a Benchmark Framework , 2015 .

[53]  Lee E. Miller,et al.  A neural population mechanism for rapid learning , 2017 .

[54]  Byron M. Yu,et al.  Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity , 2020, Nature Biomedical Engineering.

[55]  Liam Paninski,et al.  Multilayer Recurrent Network Models of Primate Retinal Ganglion Cell Responses , 2016, ICLR.

[56]  Shreya Saxena,et al.  Towards the neural population doctrine , 2019, Current Opinion in Neurobiology.

[57]  Jean-Philippe Vert,et al.  NetNorM: Capturing cancer-relevant information in somatic exome mutation data with gene networks for cancer stratification and prognosis , 2017, PLoS Comput. Biol..

[58]  T Kobayashi,et al.  Correlation dimension of the human sleep electroencephalogram , 2000, Psychiatry and clinical neurosciences.

[59]  Christopher D. Harvey,et al.  Choice-specific sequences in parietal cortex during a virtual-navigation decision task , 2012, Nature.

[60]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[61]  Matthew T. Kaufman,et al.  Supplementary materials for : Cortical activity in the null space : permitting preparation without movement , 2014 .

[62]  Francesco Camastra,et al.  Data dimensionality estimation methods: a survey , 2003, Pattern Recognit..

[63]  Sliman J. Bensmaia,et al.  Unexpected complexity of everyday manual behaviors , 2020, Nature Communications.

[64]  Jstor Journal of the Royal Statistical Society. Series A (General) , 1987 .

[65]  Lee E. Miller,et al.  A Neural Population Mechanism for Rapid Learning , 2017, Neuron.

[66]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[67]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[68]  A. Buja,et al.  Remarks on Parallel Analysis. , 1992, Multivariate behavioral research.

[69]  Lee E. Miller,et al.  Neural Manifolds for the Control of Movement , 2017, Neuron.

[70]  Abigail A. Russo,et al.  Motor Cortex Embeds Muscle-like Commands in an Untangled Population Response , 2018, Neuron.

[71]  Surya Ganguli,et al.  Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis , 2017, Neuron.

[72]  Gamaleldin F. Elsayed,et al.  Structure in neural population recordings: an expected byproduct of simpler phenomena? , 2017, Nature Neuroscience.

[73]  John P. Cunningham,et al.  Linear dynamical neural population models through nonlinear embeddings , 2016, NIPS.

[74]  Christian Ethier,et al.  Cortical population activity within a preserved neural manifold underlies multiple motor behaviors , 2018, Nature Communications.