Inferring single-trial neural population dynamics using sequential auto-encoders

Neuroscience is experiencing a revolution in which simultaneous recording of thousands of neurons is revealing population dynamics that are not apparent from single-neuron responses. This structure is typically extracted from data averaged across many trials, but deeper understanding requires studying phenomena detected in single trials, which is challenging due to incomplete sampling of the neural population, trial-to-trial variability, and fluctuations in action potential timing. We introduce latent factor analysis via dynamical systems, a deep learning method to infer latent dynamics from single-trial neural spiking data. When applied to a variety of macaque and human motor cortical datasets, latent factor analysis via dynamical systems accurately predicts observed behavioral variables, extracts precise firing rate estimates of neural dynamics on single trials, infers perturbations to those dynamics that correlate with behavioral choices, and combines data from non-overlapping recording sessions spanning months to improve inference of underlying dynamics.LFADS, a deep learning method for analyzing neural population activity, can extract neural dynamics from single-trial recordings, stitch separate datasets into a single model, and infer perturbations, for example, from behavioral choices to these dynamics.

[1]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[2]  John P. Cunningham,et al.  Empirical models of spiking in neural populations , 2011, NIPS.

[3]  Bradley C. Lega,et al.  Closed-loop stimulation of temporal cortex rescues functional networks and improves memory , 2018, Nature Communications.

[4]  Emilio Salinas,et al.  Vector reconstruction from firing rates , 1994, Journal of Computational Neuroscience.

[5]  Matthew T. Kaufman,et al.  A neural network that finds a naturalistic solution for the production of muscle activity , 2015, Nature Neuroscience.

[6]  R. Yuste.  From the neuron doctrine to neural networks , 2015, Nature Reviews Neuroscience.

[7]  N. Parga,et al.  Dynamic Control of Response Criterion in Premotor Cortex during Perceptual Detection under Temporal Uncertainty , 2015, Neuron.

[8]  Alex Graves,et al.  DRAW: A Recurrent Neural Network For Image Generation , 2015, ICML.

[9]  Byron M. Yu,et al.  Neural constraints on learning , 2014, Nature.

[10]  Matthew T. Kaufman,et al.  Supplementary materials for : Cortical activity in the null space : permitting preparation without movement , 2014 .

[11]  Chethan Pandarinath,et al.  LFADS - Latent Factor Analysis via Dynamical Systems , 2016, ArXiv.

[12]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[13]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[14]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[15]  Jakob H. Macke,et al.  Extracting low-dimensional dynamics from multiple large-scale neural population recordings by learning to predict correlations , 2017, NIPS.

[16]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[17]  Yuan Zhao,et al.  Variational Latent Gaussian Process for Recovering Single-Trial Dynamics from Population Spike Trains , 2016, Neural Computation.

[18]  Scott W. Linderman,et al.  Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems , 2017, AISTATS.

[19]  P. Brown,et al.  Adaptive Deep Brain Stimulation In Advanced Parkinson Disease , 2013, Annals of neurology.

[20]  Drew N. Robson,et al.  Brain-wide neuronal dynamics during motor adaptation in zebrafish , 2012, Nature.

[21]  Theodore H. Lindsay,et al.  Global Brain Dynamics Embed the Motor Command Sequence of Caenorhabditis elegans , 2015, Cell.

[22]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[23]  John P. Cunningham,et al.  High-dimensional neural spike train analysis with generalized count linear dynamical systems , 2015, NIPS.

[24]  L. Paninski.  Maximum likelihood estimation of cascade point-process neural encoding models , 2004, Network.

[25]  Surya Ganguli,et al.  On simplicity and complexity in the brave new world of large-scale neuroscience , 2015, Current Opinion in Neurobiology.

[26]  Byron M. Yu,et al.  Single-Trial Neural Correlates of Arm Movement Preparation , 2011, Neuron.

[27]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[28]  Chethan Pandarinath,et al.  Neural population dynamics in human motor cortex during movements in people with ALS , 2015, eLife.

[29]  Adam M. Packer,et al.  Inferring neural population dynamics from multiple partial recordings of the same neural circuit , 2013, NIPS.

[30]  Naoshige Uchida,et al.  Demixed principal component analysis of neural population data , 2014, eLife.

[31]  David Sussillo,et al.  Making brain–machine interfaces robust to future neural variability , 2016, Nature communications.

[32]  Maneesh Sahani,et al.  Equating information-theoretic and likelihood-based methods for neural dimensionality reduction , 2013, 1308.3542.

[33]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[34]  Chethan Pandarinath,et al.  Feedback control policies employed by people using intracortical brain–computer interfaces , 2017, Journal of neural engineering.

[35]  Paul Nuyujukian,et al.  Intention estimation in brain–machine interfaces , 2014, Journal of neural engineering.

[36]  John P. Cunningham,et al.  Linear dynamical neural population models through nonlinear embeddings , 2016, NIPS.

[37]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[38]  Maximilian Karl,et al.  Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data , 2016, ICLR.

[39]  P. Fries.  A mechanism for cognitive dynamics: neuronal communication through neuronal coherence , 2005, Trends in Cognitive Sciences.

[40]  Uri Shalit,et al.  Deep Kalman Filters , 2015, ArXiv.

[41]  Mehdi Aghagolzadeh,et al.  Latent state-space models for neural decoding , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[42]  Christopher D. Harvey,et al.  Recurrent Network Models of Sequence Generation and Memory , 2016, Neuron.

[43]  John P. Cunningham,et al.  Dynamical segmentation of single trials from population neural data , 2011, NIPS.

[44]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[45]  David Sussillo,et al.  A recurrent neural network for closed-loop intracortical brain–machine interface decoders , 2012, Journal of neural engineering.

[46]  Christian Osendorfer,et al.  Learning Stochastic Recurrent Networks , 2014, NIPS 2014.

[47]  Martin A. Riedmiller,et al.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images , 2015, NIPS.

[48]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[49]  Christopher D. Harvey,et al.  Choice-specific sequences in parietal cortex during a virtual-navigation decision task , 2012, Nature.

[50]  Matthew T. Kaufman,et al.  The Largest Response Component in the Motor Cortex Reflects Movement Timing but Not Movement Type , 2016, eNeuro.

[51]  S. Haber,et al.  Closed-Loop Deep Brain Stimulation Is Superior in Ameliorating Parkinsonism , 2011, Neuron.

[52]  S. Mittal,et al.  Clinical efficacy of deep brain stimulation for the treatment of medically refractory epilepsy , 2016, Clinical Neurology and Neurosurgery.

[53]  J. Donoghue,et al.  Neural discharge and local field potential oscillations in primate motor cortex during voluntary movements. , 1998, Journal of neurophysiology.

[54]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[55]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[56]  Carl Doersch,et al.  Tutorial on Variational Autoencoders , 2016, ArXiv.

[57]  E. Fetz,et al.  Synchronization of neurons during local field potential oscillations in sensorimotor cortex of awake monkeys. , 1996, Journal of neurophysiology.

[58]  M. Sahani,et al.  Cortical control of arm movements: a dynamical systems perspective. , 2013, Annual review of neuroscience.

[59]  Chethan Pandarinath,et al.  High performance communication by people with paralysis using an intracortical brain-computer interface , 2017, eLife.

[60]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[61]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[62]  John P. Cunningham,et al.  Single-trial dynamics of motor cortex and their applications to brain-machine interfaces , 2015, Nature Communications.