Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor components analysis

Perceptions, thoughts and actions unfold over millisecond timescales, while learned behaviors can require many days to mature. While recent experimental advances enable large-scale and long-term neural recordings with high temporal fidelity, it remains a formidable challenge to extract unbiased and interpretable descriptions of how rapid single-trial circuit dynamics change slowly over many trials to mediate learning. We demonstrate a simple tensor components analysis (TCA) can meet this challenge by extracting three interconnected low dimensional descriptions of neural data: neuron factors, reflecting cell assemblies; temporal factors, reflecting rapid circuit dynamics mediating perceptions, thoughts, and actions within each trial; and trial factors, describing both long-term learning and trial-to-trial changes in cognitive state. We demonstrate the broad applicability of TCA by revealing insights into diverse datasets derived from artificial neural networks, large-scale calcium imaging of rodent prefrontal cortex during maze navigation, and multielectrode recordings of macaque motor cortex during brain machine interface learning.

[1]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[2]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[3]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[4]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[5]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[6]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[7]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[9]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[10]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[11]  H S Seung,et al.  How the brain keeps the eyes still. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[12]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[13]  P. Paatero A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[14]  J. Kleim,et al.  Functional reorganization of the rat motor cortex following motor skill learning. , 1998, Journal of neurophysiology.

[15]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[16]  Emilio Salinas,et al.  Gain Modulation A Major Computational Principle of the Central Nervous System , 2000, Neuron.

[17]  Max Welling,et al.  Positive tensor factorization , 2001, Pattern Recognit. Lett..

[18]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[19]  Frances S. Chance,et al.  Gain Modulation from Background Synaptic Input , 2002, Neuron.

[20]  Emery N. Brown,et al.  Estimating a State-space Model from Point Process Observations Emery N. Brown , 2022 .

[21]  Tamara G. Kolda,et al.  A Counterexample to the Possibility of an Extension of the Eckart-Young Low-Rank Approximation Theorem for the Orthogonal Rank Tensor Decomposition , 2002, SIAM J. Matrix Anal. Appl..

[22]  S. Prescott,et al.  Gain control of firing rate by shunting inhibition: Roles of synaptic noise and dendritic saturation , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[23]  William S Rayens,et al.  Structure-seeking multilinear methods for the analysis of fMRI data , 2004, NeuroImage.

[24]  I. Dean,et al.  Neural population coding of sound level adapts to stimulus statistics , 2005, Nature Neuroscience.

[25]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[26]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[27]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[28]  Lars Kai Hansen,et al.  Parallel Factor Analysis as an exploratory tool for wavelet transformed event-related EEG , 2006, NeuroImage.

[29]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[30]  G. Golub,et al.  A tensor higher-order singular value decomposition for integrative analysis of DNA microarray data from different studies , 2007, Proceedings of the National Academy of Sciences.

[31]  Rasmus Bro,et al.  Multiway analysis of epilepsy tensors , 2007, ISMB/ECCB.

[32]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[33]  M. Sahani,et al.  Nonlinearities and Contextual Influences in Auditory Cortical Responses Modeled with Multilinear Spectrotemporal Methods , 2008, The Journal of Neuroscience.

[34]  Jessica A. Cardin,et al.  Cellular Mechanisms Underlying Stimulus-Dependent Gain Modulation in Primary Visual Cortex Neurons In Vivo , 2008, Neuron.

[35]  C. Law,et al.  Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area , 2008, Nature Neuroscience.

[36]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[37]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[38]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[39]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[40]  Bernd Sturmfels,et al.  Reconstructing spatiotemporal gene expression data from partial observations , 2009, Bioinform..

[41]  Erin L. Rich,et al.  Rat Prefrontal Cortical Neurons Selectively Code Strategy Switches , 2009, The Journal of Neuroscience.

[42]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[43]  J. White,et al.  Gain Control in CA1 Pyramidal Cells Using Changes in Somatic Conductance , 2010, The Journal of Neuroscience.

[44]  S. Kennerley,et al.  Heterogeneous reward signals in prefrontal cortex , 2010, Current Opinion in Neurobiology.

[45]  M. Stryker,et al.  Modulation of Visual Responses by Behavioral State in Mouse Visual Cortex , 2010, Neuron.

[46]  D. Durstewitz,et al.  Abrupt Transitions between Prefrontal Neural Ensemble States Accompany Behavioral Transitions during Rule Learning , 2010, Neuron.

[47]  J. Maunsell,et al.  A Neuronal Population Measure of Attention Predicts Behavioral Performance on Individual Trials , 2010, The Journal of Neuroscience.

[48]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[49]  A. Gamal,et al.  Miniaturized integration of a fluorescence microscope , 2011, Nature Methods.

[50]  Johannes J. Letzkus,et al.  A disinhibitory microcircuit for associative fear learning in the auditory cortex , 2011, Nature.

[51]  Vittorio Ferrari,et al.  Advances in Neural Information Processing Systems 24 , 2011 .

[52]  Haesun Park,et al.  Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons , 2011, SIAM J. Sci. Comput..

[53]  J. Maunsell,et al.  When Attention Wanders: How Uncontrolled Fluctuations in Attention Affect Performance , 2011, The Journal of Neuroscience.

[54]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[55]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[56]  M. Carandini,et al.  Parvalbumin-Expressing Interneurons Linearly Transform Cortical Responses to Visual Stimuli , 2012, Neuron.

[57]  Maneesh Sahani,et al.  Spectral learning of linear dynamics from generalised-linear observations with application to neural population data , 2012, NIPS.

[58]  Tamara G. Kolda,et al.  On Tensors, Sparsity, and Nonnegative Factorizations , 2011, SIAM J. Matrix Anal. Appl..

[59]  T. Komiyama,et al.  Parvalbumin-Expressing Interneurons Linearly Control Olfactory Bulb Output , 2013, Neuron.

[60]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[61]  M. Sahani,et al.  Cortical control of arm movements: a dynamical systems perspective. , 2013, Annual review of neuroscience.

[62]  Jan M. Rabaey,et al.  Physical principles for scalable neural recording , 2013, Front. Comput. Neurosci..

[63]  N. Sigala,et al.  Dynamic Coding for Cognitive Control in Prefrontal Cortex , 2013, Neuron.

[64]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[65]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[66]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[67]  Dean V. Buonomano,et al.  ROBUST TIMING AND MOTOR PATTERNS BY TAMING CHAOS IN RECURRENT NEURAL NETWORKS , 2012, Nature Neuroscience.

[68]  F. Helmchen,et al.  Steady or changing? Long-term monitoring of neuronal population activity , 2013, Trends in Neurosciences.

[69]  Eero P. Simoncelli,et al.  Partitioning neuronal variability , 2014, Nature Neuroscience.

[70]  David Sussillo,et al.  Neural circuits as computational dynamical systems , 2014, Current Opinion in Neurobiology.

[71]  Aaron C. Koralek,et al.  Volitional modulation of optically recorded calcium signals during neuroprosthetic learning , 2014, Nature Neuroscience.

[72]  Simon X. Chen,et al.  Emergence of reproducible spatiotemporal activity during motor learning , 2014, Nature.

[73]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[74]  John P. Cunningham,et al.  Single-trial dynamics of motor cortex and their applications to brain-machine interfaces , 2015, Nature Communications.

[75]  Eero P. Simoncelli,et al.  Attention stabilizes the shared gain of V4 , 2015 .

[76]  Xiaofeng Gong,et al.  Tensor decomposition of EEG signals: A brief review , 2015, Journal of Neuroscience Methods.

[77]  Eero P. Simoncelli,et al.  Attention stabilizes the shared gain of V4 populations , 2015, eLife.

[78]  Rasmus Bro,et al.  Data Fusion in Metabolomics Using Coupled Matrix and Tensor Factorizations , 2015, Proceedings of the IEEE.

[79]  Surya Ganguli,et al.  On simplicity and complexity in the brave new world of large-scale neuroscience , 2015, Current Opinion in Neurobiology.

[80]  Guangyu R. Yang,et al.  Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework , 2016, PLoS Comput. Biol..

[81]  John P. Cunningham,et al.  Tensor Analysis Reveals Distinct Population Structure that Parallels the Different Computational Roles of Areas M1 and V1 , 2016, PLoS Comput. Biol..

[82]  Yuan Zhao,et al.  Interpretable Nonlinear Dynamic Modeling of Neural Trajectories , 2016, NIPS.

[83]  Michael Z. Lin,et al.  Genetically encoded indicators of neuronal activity , 2016, Nature Neuroscience.

[84]  Naoshige Uchida,et al.  Demixed principal component analysis of neural population data , 2016, eLife.

[85]  Scott W. Linderman,et al.  Recurrent switching linear dynamical systems , 2016, 1610.08466.

[86]  Rajesh Poddar,et al.  Automated long-term recording and analysis of neural activity in behaving animals , 2016 .

[87]  T. Komiyama,et al.  Circuit Mechanisms of Sensorimotor Learning , 2016, Neuron.

[88]  Hongkui Zeng,et al.  Long-Term Optical Access to an Estimated One Million Neurons in the Live Mouse Cortex. , 2016, Cell reports.

[89]  M. Siniscalchi,et al.  Fast and slow transitions in frontal ensemble activity during flexible sensorimotor behavior , 2016, Nature Neuroscience.

[90]  M. McCarthy,et al.  Tensor decomposition for multi-tissue gene expression experiments , 2016, Nature Genetics.

[91]  John P. Cunningham,et al.  Linear dynamical neural population models through nonlinear embeddings , 2016, NIPS.

[92]  Mario Dipoppa,et al.  Suite2p: beyond 10,000 neurons with standard two-photon microscopy , 2016, bioRxiv.

[93]  Pablo E. Jercog,et al.  Neural ensemble dynamics underlying a long-term associative memory , 2017, Nature.

[94]  Xiao-Jing Wang,et al.  Reward-based training of recurrent neural networks for cognitive and value-based tasks , 2016, bioRxiv.

[95]  Patrick Dupont,et al.  Tensor decompositions and data fusion in epileptic electroencephalography and functional magnetic resonance imaging data , 2017, WIREs Data Mining Knowl. Discov..

[96]  Polina Anikeeva,et al.  Neural Recording and Modulation Technologies. , 2017, Nature reviews. Materials.

[97]  Euisik Yoon,et al.  State-of-the-art MEMS and microsystem tools for brain research , 2017, Microsystems & Nanoengineering.

[98]  Selmaan N. Chettih,et al.  Dynamic Reorganization of Neuronal Activity Patterns in Parietal Cortex , 2017, Cell.

[99]  Konrad Paul Kording,et al.  Could a Neuroscientist Understand a Microprocessor? , 2016, bioRxiv.

[100]  Chethan Pandarinath,et al.  Inferring single-trial neural population dynamics using sequential auto-encoders , 2017 .

[101]  Scott W. Linderman,et al.  Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems , 2017, AISTATS.

[102]  Chethan Pandarinath,et al.  Inferring single-trial neural population dynamics using sequential auto-encoders , 2017, Nature Methods.

[103]  Maja Pantic,et al.  TensorLy: Tensor Learning in Python , 2016, J. Mach. Learn. Res..