Brain–Machine Interfaces

Motor brain–machine interfaces (BMIs) allow subjects to control external devices by modulating their neural activity. BMIs record motor cortical activities, use a decoding algorithm to infer the subject’s intended movement and control a prosthetic device, and provide visual feedback to the subject. Thus BMIs can be viewed as closed-loop control systems. In this chapter, we review the computational components of a BMI and the common decoders used in the field. We then discuss in detail a recent control-theoretic high-rate BMI decoder, termed adaptive optimal feedback-controlled point process filter (OFC-PPF), which has significantly improved performance and robustness. This decoder characterizes the spikes directly using a point process model and learns the model parameters using closed-loop decoder adaptation. The decoder also models the BMI as an optimal feedback-control system to better infer the brain’s intention during adaptation. This decoder significantly improves the speed and accuracy of model adaptation. Moreover, at steady state, the learned point process filter improves performance over the state-of-the-art Kalman filters due to the fast control and feedback rates and the point process encoding model.

[1]  E. Fetz Operant Conditioning of Cortical Unit Activity , 1969, Science.

[2]  D R Humphrey,et al.  Predicting Measures of Motor Performance from Multiple Cortical Spike Trains , 1970, Science.

[3]  E N Brown,et al.  A Statistical Paradigm for Neural Spike Train Decoding Applied to Position Prediction from Ensemble Firing Patterns of Rat Hippocampal Place Cells , 1998, The Journal of Neuroscience.

[4]  Miguel A. L. Nicolelis,et al.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex , 1999, Nature Neuroscience.

[5]  A B Schwartz,et al.  Motor cortical representation of speed and direction during reaching. , 1999, Journal of neurophysiology.

[6]  Scott T. Grafton,et al.  Forward modeling allows feedback control for fast reaching movements , 2000, Trends in Cognitive Sciences.

[7]  Robert E. Kass,et al.  A Spike-Train Probability Model , 2001, Neural Computation.

[8]  Emery N. Brown,et al.  The Time-Rescaling Theorem and Its Application to Neural Spike Train Data Analysis , 2002, Neural Computation.

[9]  Nicholas G. Hatsopoulos,et al.  Brain-machine interface: Instant neural control of a movement signal , 2002, Nature.

[10]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[11]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[12]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[13]  R E Kass,et al.  Recursive bayesian decoding of motor cortical signals by particle filtering. , 2004, Journal of neurophysiology.

[14]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[15]  Emery N. Brown,et al.  Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering , 2004, Neural Computation.

[16]  Uri T Eden,et al.  A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. , 2005, Journal of neurophysiology.

[17]  Emery N. Brown,et al.  A State-Space Analysis for Reconstruction of Goal-Directed Movements Using Neural Signals , 2006, Neural Computation.

[18]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[19]  Byron M. Yu,et al.  A high-performance brain–computer interface , 2006, Nature.

[20]  Byron M. Yu,et al.  Mixture of Trajectory Models for Neural Decoding of Goal-directed Movements a Computational Model of Craving and Obsession Decoding Visual Inputs from Multiple Neurons in the Human Temporal Lobe Encoding Contribution of Individual Retinal Ganglion Cell Responses to Velocity and Acceleration , 2008 .

[21]  Richard A Andersen,et al.  Decoding Trajectories from Posterior Parietal Cortex Ensembles , 2008, The Journal of Neuroscience.

[22]  E. Fetz,et al.  Direct control of paralyzed muscles by cortical neurons , 2008, Nature.

[23]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.

[24]  Robert E Kass,et al.  Functional network reorganization during learning in a brain-computer interface paradigm , 2008, Proceedings of the National Academy of Sciences.

[25]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[26]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[27]  Robert E. Kass,et al.  2009 Special Issue: Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms , 2009 .

[28]  Joseph E. O’Doherty,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[29]  Robert E. Kass,et al.  Comparison of brain–computer interface decoding algorithms in open-loop and closed-loop control , 2010, Journal of Computational Neuroscience.

[30]  B. McClane,et al.  Use of an EZ-Tn5-Based Random Mutagenesis System to Identify a Novel Toxin Regulatory Locus in Clostridium perfringens Strain 13 , 2009, PloS one.

[31]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[32]  Nicholas G Hatsopoulos,et al.  Incorporating Feedback from Multiple Sensory Modalities Enhances Brain–Machine Interface Control , 2010, The Journal of Neuroscience.

[33]  Miguel A. L. Nicolelis,et al.  Adaptive Decoding for Brain-Machine Interfaces Through Bayesian Parameter Updates , 2011, Neural Computation.

[34]  Hannes Bleuler,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[35]  Justin C. Sanchez,et al.  A Symbiotic Brain-Machine Interface through Value-Based Decision Making , 2011, PloS one.

[36]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[37]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[38]  Emery N. Brown,et al.  Neural population partitioning and a concurrent brain-machine interface for sequential motor function , 2012, Nature Neuroscience.

[39]  A. Schwartz,et al.  Behavioral and neural correlates of visuomotor adaptation observed through a brain-computer interface in primary motor cortex. , 2012, Journal of neurophysiology.

[40]  J. M. Carmena,et al.  Closed-Loop Decoder Adaptation on Intermediate Time-Scales Facilitates Rapid BMI Performance Improvements Independent of Decoder Initialization Conditions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[41]  L. Miller,et al.  Restoration of grasp following paralysis through brain-controlled stimulation of muscles , 2012, Nature.

[42]  Gerald E Loeb,et al.  Cognitive signals for brain–machine interfaces in posterior parietal cortex include continuous 3D trajectory commands , 2012, Proceedings of the National Academy of Sciences.

[43]  Byron M. Yu,et al.  Internal models engaged by brain-computer interface control , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[44]  N. V. Thakor,et al.  Translating the Brain-Machine Interface , 2013, Science Translational Medicine.

[45]  Ziv M. Williams,et al.  A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design , 2013, PloS one.

[46]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[47]  Nicholas G Hatsopoulos,et al.  Improving brain–machine interface performance by decoding intended future movements , 2013, Journal of neural engineering.

[48]  G. W. Wornell,et al.  Feedback-Controlled Parallel Point Process Filter for Estimation of Goal-Directed Movements From Neural Signals , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[49]  Bryan D. He,et al.  Dynamic Analysis of Naive Adaptive Brain-Machine Interfaces , 2013, Neural Computation.

[50]  Jose M. Carmena,et al.  Continuous Closed-Loop Decoder Adaptation with a Recursive Maximum Likelihood Algorithm Allows for Rapid Performance Acquisition in Brain-Machine Interfaces , 2014, Neural Computation.

[51]  Paul Nuyujukian,et al.  Intention estimation in brain–machine interfaces , 2014, Journal of neural engineering.

[52]  Jose M. Carmena,et al.  Closed-Loop Decoder Adaptation Shapes Neural Plasticity for Skillful Neuroprosthetic Control , 2014, Neuron.

[53]  Nitish V. Thakor,et al.  Demonstration of a Semi-Autonomous Hybrid Brain–Machine Interface Using Human Intracranial EEG, Eye Tracking, and Computer Vision to Control a Robotic Upper Limb Prosthetic , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[54]  Ziv M. Williams,et al.  A cortical-spinal prosthesis for targeted limb movement in paralyzed primate avatars , 2014, Nature Communications.

[55]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[56]  Maryam Modir Shanechi,et al.  Optimal calibration of the learning rate in closed-loop adaptive brain-machine interfaces , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[57]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[58]  John P. Cunningham,et al.  Encoder-Decoder Optimization for Brain-Computer Interfaces , 2015, PLoS Comput. Biol..

[59]  Maryam Modir Shanechi,et al.  Multiscale brain-machine interface decoders , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[60]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[61]  Qin,et al.  A Brain–Spinal Interface Alleviating Gait Deficits after Spinal Cord Injury in Primates , 2017 .

[62]  Jose M. Carmena,et al.  Robust Brain-Machine Interface Design Using Optimal Feedback Control Modeling and Adaptive Point Process Filtering , 2016, PLoS Comput. Biol..

[63]  Maryam Modir Shanechi,et al.  An unsupervised learning algorithm for multiscale neural activity , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[64]  Bijan Pesaran,et al.  Multiscale decoding for reliable brain-machine interface performance over time , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[65]  Jose M. Carmena,et al.  Rapid control and feedback rates enhance neuroprosthetic control , 2017, Nature Communications.