Adversarial Domain Adaptation for Stable Brain-Machine Interfaces

Brain-Machine Interfaces (BMIs) have recently emerged as a clinically viable option to restore voluntary movements after paralysis. These devices are based on the ability to extract information about movement intent from neural signals recorded using multi-electrode arrays chronically implanted in the motor cortices of the brain. However, the inherent loss and turnover of recorded neurons requires repeated recalibrations of the interface, which can potentially alter the day-to-day user experience. The resulting need for continued user adaptation interferes with the natural, subconscious use of the BMI. Here, we introduce a new computational approach that decodes movement intent from a low-dimensional latent representation of the neural data. We implement various domain adaptation methods to stabilize the interface over significantly long times. This includes Canonical Correlation Analysis used to align the latent variables across days; this method requires prior point-to-point correspondence of the time series across domains. Alternatively, we match the empirical probability distributions of the latent variables across days through the minimization of their Kullback-Leibler divergence. These two methods provide a significant and comparable improvement in the performance of the interface. However, implementation of an Adversarial Domain Adaptation Network trained to match the empirical probability distribution of the residuals of the reconstructed neural signals outperforms the two methods based on latent variables, while requiring remarkably few data points to solve the domain adaptation problem.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[3]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[4]  K. Anderson Targeting recovery: priorities of the spinal cord-injured population. , 2004, Journal of neurotrauma.

[5]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[6]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[7]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[8]  Yali Amit,et al.  Single-unit stability using chronically implanted multielectrode arrays. , 2009, Journal of neurophysiology.

[9]  J. M. Carmena,et al.  Closed-Loop Decoder Adaptation on Intermediate Time-Scales Facilitates Rapid BMI Performance Improvements Independent of Decoder Initialization Conditions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[10]  L. Miller,et al.  Restoration of grasp following paralysis through brain-controlled stimulation of muscles , 2012, Nature.

[11]  Lee E Miller,et al.  Movement representation in the primary motor cortex and its contribution to generalizable EMG predictions. , 2013, Journal of neurophysiology.

[12]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[13]  M. Sahani,et al.  Cortical control of arm movements: a dynamical systems perspective. , 2013, Annual review of neuroscience.

[14]  J. Donoghue,et al.  Failure mode analysis of silicon-based intracortical microelectrode arrays in non-human primates , 2013, Journal of neural engineering.

[15]  Yin Zhang,et al.  A stabilized dual Kalman filter for adaptive tracking of brain-computer interface decoding parameters , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Jose M. Carmena,et al.  Design and Analysis of Closed-Loop Decoder Adaptation Algorithms for Brain-Machine Interfaces , 2013, Neural Computation.

[17]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[18]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[19]  Paul Nuyujukian,et al.  Performance sustaining intracortical neural prostheses , 2014, Journal of neural engineering.

[20]  Byron M. Yu,et al.  Self-recalibrating classifiers for intracortical brain–computer interfaces , 2014, Journal of neural engineering.

[21]  Byron M. Yu,et al.  Neural constraints on learning , 2014, Nature.

[22]  Vikash Gilja,et al.  Assessment of brain–machine interfaces from the perspective of people with paralysis , 2015, Journal of neural engineering.

[23]  Nicolas Y. Masse,et al.  Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface , 2015, Science Translational Medicine.

[24]  Matthew T. Kaufman,et al.  A neural network that finds a naturalistic solution for the production of muscle activity , 2015, Nature Neuroscience.

[25]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[26]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[27]  David Sussillo,et al.  Making brain–machine interfaces robust to future neural variability , 2016, Nature communications.

[28]  Stephen I. Ryu,et al.  Leveraging neural dynamics to extend functional lifetime of brain-machine interfaces , 2017, Scientific Reports.

[29]  Yoshua Bengio,et al.  Improving Generative Adversarial Networks with Denoising Feature Matching , 2016, ICLR.

[30]  Lee E. Miller,et al.  Neural Manifolds for the Control of Movement , 2017, Neuron.

[31]  David Berthelot,et al.  BEGAN: Boundary Equilibrium Generative Adversarial Networks , 2017, ArXiv.

[32]  Francis R. Willett,et al.  Restoration of reaching and grasping in a person with tetraplegia through brain-controlled muscle stimulation: a proof-of-concept demonstration , 2017, The Lancet.

[33]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[34]  Eva L. Dyer,et al.  A cryptography-based approach for movement decoding , 2016, Nature Biomedical Engineering.

[35]  Lee E. Miller,et al.  Multiple tasks viewed from the neural manifold: Stable control of varied behavior , 2017, bioRxiv.

[36]  Chethan Pandarinath,et al.  Inferring single-trial neural population dynamics using sequential auto-encoders , 2017, Nature Methods.

[37]  Steven M Chase,et al.  Intracortical recording stability in human brain–computer interface users , 2018, Journal of neural engineering.

[38]  Abigail A. Russo,et al.  Motor Cortex Embeds Muscle-like Commands in an Untangled Population Response , 2018, Neuron.

[39]  Christian Ethier,et al.  Cortical population activity within a preserved neural manifold underlies multiple motor behaviors , 2018, Nature Communications.