Interactions of encoding and decoding problems to understand motor control

Learning a map from movement to neural data (Encoding Problem) and vice versa (Decoding Problem) are crucial to understanding motor control. A principled encoding model that understands underlying neural dynamics can help better solve the decoding problem. Here, we develop a new generative encoding model leveraging deep learning that autonomously captures neural dynamics. After training, the model can synthesize spike trains given any observed kinematics, under the guidance of the learned neural dynamics. When neural data from other sessions or subjects are limited, synthesized spike trains can improve cross-session and cross-subject decoding performance of a Brain Computer Interface decoder. For cross-subject, even with ample data for both subjects, neural dynamics learned from a previous subject can transfer useful knowledge that improves the best achievable decoding performance for the new subject. The approach is general and fully data-driven, and hence could apply to neuroscience encoding and decoding problems beyond motor control.

[1]  Hugues Duffau,et al.  Brain Plasticity and Reorganization Before, During, and After Glioma Resection , 2016 .

[2]  Takeru Miyato,et al.  cGANs with Projection Discriminator , 2018, ICLR.

[3]  Reza Shadmehr,et al.  Learning of action through adaptive combination of motor primitives , 2000, Nature.

[4]  Konrad P. Körding,et al.  Machine Learning for Neural Decoding , 2017, eNeuro.

[5]  Yann Le Cun,et al.  A Theoretical Framework for Back-Propagation , 1988 .

[6]  Uri T Eden,et al.  A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. , 2005, Journal of neurophysiology.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  L. Paninski,et al.  Superlinear Population Encoding of Dynamic Hand Trajectory in Primary Motor Cortex , 2004, The Journal of Neuroscience.

[9]  Kristian Franze,et al.  The soft mechanical signature of glial scars in the central nervous system , 2017, Nature Communications.

[10]  Franco Turini,et al.  Time-Annotated Sequences for Medical Data Mining , 2007 .

[11]  Emery N. Brown,et al.  Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering , 2004, Neural Computation.

[12]  Miguel A. L. Nicolelis,et al.  Brain–machine interfaces: past, present and future , 2006, Trends in Neurosciences.

[13]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[14]  Chethan Pandarinath,et al.  Inferring single-trial neural population dynamics using sequential auto-encoders , 2017, Nature Methods.

[15]  Michael J. Black,et al.  A quantitative comparison of linear and non-linear models of motor cortical activity for the encoding and decoding of arm motions , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[16]  Thibaut Horel,et al.  Modeling Contagion Through Social Networks to Explain and Predict Gunshot Violence in Chicago, 2006 to 2014 , 2017, JAMA internal medicine.

[17]  Liam Paninski,et al.  Statistical models for neural encoding, decoding, and optimal stimulus design. , 2007, Progress in brain research.

[18]  J. Goodier.  The Concise Encyclopedia of Statistics , 2009 .

[19]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[20]  Guillaume Hennequin,et al.  Motor primitives in space and time via targeted gain modulation in cortical networks , 2018, Nature Neuroscience.

[21]  F. van der Meer,et al.  Induction of Toll-like receptor 4 signaling in avian macrophages inhibits infectious laryngotracheitis virus replication in a nitric oxide dependent way. , 2013, Veterinary immunology and immunopathology.

[22]  Grgoire Montavon,et al.  Neural Networks: Tricks of the Trade , 2012, Lecture Notes in Computer Science.

[23]  R E Kass,et al.  Recursive bayesian decoding of motor cortical signals by particle filtering. , 2004, Journal of neurophysiology.

[24]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[25]  Dennis C. Tkach,et al.  Observation-based learning for brain–machine interfaces , 2008, Current Opinion in Neurobiology.

[26]  Uri T. Eden,et al.  Point process adaptive filters for neural data analysis: Theory and applications , 2007, 2007 46th IEEE Conference on Decision and Control.

[27]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[28]  Wei Wu,et al.  Neural Decoding of Cursor Motion Using a Kalman Filter , 2002, NIPS.

[29]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[30]  S. Schaal.  Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[31]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[32]  A. Michael,et al.  Brain Tissue Responses to Neural Implants Impact Signal Sensitivity and Intervention Strategies , 2014, ACS chemical neuroscience.

[33]  Tomaso Poggio,et al.  Generalization in vision and motor control , 2004, Nature.

[34]  Augustus Odena,et al.  Semi-Supervised Learning with Generative Adversarial Networks , 2016, ArXiv.

[35]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[36]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[37]  Gidon Felsen,et al.  A natural approach to studying vision , 2005, Nature Neuroscience.

[38]  Paul Nuyujukian,et al.  Performance sustaining intracortical neural prostheses , 2014, Journal of neural engineering.

[39]  Georg Heigold,et al.  An empirical study of learning rates in deep neural networks for speech recognition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[40]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[41]  M. Nicolelis,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[42]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[43]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[44]  Ramesh Nallapati,et al.  A Comparative Study of Methods for Transductive Transfer Learning , 2007, Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007).

[45]  Eva L. Dyer,et al.  A cryptography-based approach for movement decoding , 2016, Nature Biomedical Engineering.

[46]  Joshua I Glaser,et al.  Population coding of conditional probability distributions in dorsal premotor cortex , 2017, Nature Communications.