Brain-Machine Interface Control Algorithms.

Motor brain-machine interfaces (BMI) allow subjects to control external devices by modulating their neural activity. BMIs record the neural activity, use a mathematical algorithm to estimate the subject's intended movement, actuate an external device, and provide visual feedback of the generated movement to the subject. A critical component of a BMI system is the control algorithm, termed decoder. Significant progress has been made in the design of BMI decoders in recent years resulting in proficient control in non-human primates and humans. In this review article, we discuss the decoding algorithms developed in the BMI field, with particular focus on recent designs that are informed by closed-loop control ideas. A motor BMI can be modeled as a closed-loop control system, where the controller is the brain, the plant is the prosthetic, the feedback is the biofeedback, and the control command is the neural activity. Additionally, compared to other closed-loop systems, BMIs have various unique properties. Neural activity is noisy and stochastic, and often consists of a sequence of spike trains. Neural representations of movement could be non-stationary and change over time, for example as a result of learning. We review recent decoder designs that take these unique properties into account. We also discuss the opportunities that exist at the interface of control theory, statistical inference, and neuroscience to devise a control-theoretic framework for BMI design and help develop the next-generation BMI control algorithms.

[1]  Paul Nuyujukian,et al.  Intention estimation in brain–machine interfaces , 2014, Journal of neural engineering.

[2]  R E Kass,et al.  Recursive bayesian decoding of motor cortical signals by particle filtering. , 2004, Journal of neurophysiology.

[3]  Emery N. Brown,et al.  The Time-Rescaling Theorem and Its Application to Neural Spike Train Data Analysis , 2002, Neural Computation.

[4]  G. W. Wornell,et al.  Feedback-Controlled Parallel Point Process Filter for Estimation of Goal-Directed Movements From Neural Signals , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[6]  A. Schwartz,et al.  Behavioral and neural correlates of visuomotor adaptation observed through a brain-computer interface in primary motor cortex. , 2012, Journal of neurophysiology.

[7]  Emery N. Brown,et al.  A parallel point-process filter for estimation of goal-directed movements from neural signals , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[9]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[10]  Wei Wu,et al.  Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter , 2006, Neural Computation.

[11]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[12]  J. Wolpaw,et al.  Decoding two-dimensional movement trajectories using electrocorticographic signals in humans , 2007, Journal of neural engineering.

[13]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[14]  Maryam Modir Shanechi,et al.  Multiscale brain-machine interface decoders , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[15]  Maryam Modir Shanechi,et al.  Optimal calibration of the learning rate in closed-loop adaptive brain-machine interfaces , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  L R Hochberg,et al.  Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  Jose M. Carmena,et al.  Robust Brain-Machine Interface Design Using Optimal Feedback Control Modeling and Adaptive Point Process Filtering , 2016, PLoS Comput. Biol..

[18]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[19]  Hannes Bleuler,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[20]  Byron M. Yu,et al.  Internal models for interpreting neural population activity during sensorimotor control , 2015, eLife.

[21]  Robin C. Ashmore,et al.  An Electrocorticographic Brain Interface in an Individual with Tetraplegia , 2013, PloS one.

[22]  Nitish V. Thakor,et al.  Demonstration of a Semi-Autonomous Hybrid Brain–Machine Interface Using Human Intracranial EEG, Eye Tracking, and Computer Vision to Control a Robotic Upper Limb Prosthetic , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  D R Humphrey,et al.  Predicting Measures of Motor Performance from Multiple Cortical Spike Trains , 1970, Science.

[24]  Hagai Bergman,et al.  Inducing Gamma Oscillations and Precise Spike Synchrony by Operant Conditioning via Brain-Machine Interface , 2013, Neuron.

[25]  Emery N. Brown,et al.  Neural population partitioning and a concurrent brain-machine interface for sequential motor function , 2012, Nature Neuroscience.

[26]  Gytis Baranauskas,et al.  What limits the performance of current invasive brain machine interfaces? , 2014, Front. Syst. Neurosci..

[27]  Emery N. Brown,et al.  A State-Space Analysis for Reconstruction of Goal-Directed Movements Using Neural Signals , 2006, Neural Computation.

[28]  Qin,et al.  A Brain–Spinal Interface Alleviating Gait Deficits after Spinal Cord Injury in Primates , 2017 .

[29]  Gerald E Loeb,et al.  Cognitive signals for brain–machine interfaces in posterior parietal cortex include continuous 3D trajectory commands , 2012, Proceedings of the National Academy of Sciences.

[30]  Joseph E. O’Doherty,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[31]  Emanuel Todorov,et al.  Stochastic Optimal Control and Estimation Methods Adapted to the Noise Characteristics of the Sensorimotor System , 2005, Neural Computation.

[32]  N. V. Thakor,et al.  Translating the Brain-Machine Interface , 2013, Science Translational Medicine.

[33]  Robert D Flint,et al.  Long term, stable brain machine interface performance using local field potentials and multiunit spikes , 2013, Journal of neural engineering.

[34]  Kelvin So,et al.  Subject-specific modulation of local field potential spectral power during brain–machine interface control in primates , 2014, Journal of neural engineering.

[35]  Nicholas G Hatsopoulos,et al.  Incorporating Feedback from Multiple Sensory Modalities Enhances Brain–Machine Interface Control , 2010, The Journal of Neuroscience.

[36]  Miguel A. L. Nicolelis,et al.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex , 1999, Nature Neuroscience.

[37]  E. Todorov.  Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[38]  Robert E Kass,et al.  Functional network reorganization during learning in a brain-computer interface paradigm , 2008, Proceedings of the National Academy of Sciences.

[39]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[40]  Bijan Pesaran,et al.  Optimizing the Decoding of Movement Goals from Local Field Potentials in Macaque Cortex , 2011, The Journal of Neuroscience.

[41]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[42]  Uri T Eden,et al.  A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. , 2005, Journal of neurophysiology.

[43]  Jose M. Carmena,et al.  Closed-Loop Decoder Adaptation Shapes Neural Plasticity for Skillful Neuroprosthetic Control , 2014, Neuron.

[44]  Robert D Flint,et al.  Local field potentials allow accurate decoding of muscle activity. , 2012, Journal of neurophysiology.

[45]  Emery N. Brown,et al.  Estimating a State-space Model from Point Process Observations Emery N. Brown , 2022 .

[46]  L. Miller,et al.  Restoration of grasp following paralysis through brain-controlled stimulation of muscles , 2012, Nature.

[47]  Eun Jung Hwang,et al.  Brain Control of Movement Execution Onset Using Local Field Potentials in Posterior Parietal Cortex , 2009, The Journal of Neuroscience.

[48]  E. Fetz,et al.  Direct control of paralyzed muscles by cortical neurons , 2008, Nature.

[49]  Richard A Andersen,et al.  Decoding Trajectories from Posterior Parietal Cortex Ensembles , 2008, The Journal of Neuroscience.

[50]  Bryan D. He,et al.  Dynamic Analysis of Naive Adaptive Brain-Machine Interfaces , 2013, Neural Computation.

[51]  J. M. Carmena,et al.  Closed-Loop Decoder Adaptation on Intermediate Time-Scales Facilitates Rapid BMI Performance Improvements Independent of Decoder Initialization Conditions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[52]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[53]  E N Brown,et al.  A Statistical Paradigm for Neural Spike Train Decoding Applied to Position Prediction from Ensemble Firing Patterns of Rat Hippocampal Place Cells , 1998, The Journal of Neuroscience.

[54]  Lakshminarayan Srinivasan,et al.  Stochastic Optimal Control as a Theory of Brain-Machine Interface Operation , 2013, Neural Computation.

[55]  Jose M. Carmena,et al.  Rapid control and feedback rates enhance neuroprosthetic control , 2017, Nature Communications.

[56]  Nicholas G. Hatsopoulos,et al.  Brain-machine interface: Instant neural control of a movement signal , 2002, Nature.

[57]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[58]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[59]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[60]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[61]  Byron M. Yu,et al.  Mixture of Trajectory Models for Neural Decoding of Goal-directed Movements a Computational Model of Craving and Obsession Decoding Visual Inputs from Multiple Neurons in the Human Temporal Lobe Encoding Contribution of Individual Retinal Ganglion Cell Responses to Velocity and Acceleration , 2008 .

[62]  Emery N. Brown,et al.  Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering , 2004, Neural Computation.

[63]  C. Mehring,et al.  Encoding of Movement Direction in Different Frequency Ranges of Motor Cortical Local Field Potentials , 2005, The Journal of Neuroscience.

[64]  Arjun K. Bansal,et al.  Relationships among low-frequency local field potentials, spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. , 2011, Journal of neurophysiology.

[65]  Abdullah Akce,et al.  A Feedback Information-Theoretic Approach to the Design of Brain–Computer Interfaces , 2010, Int. J. Hum. Comput. Interact..

[66]  Byron M. Yu,et al.  A high-performance brain–computer interface , 2006, Nature.

[67]  Emanuel Todorov,et al.  Evidence for the Flexible Sensorimotor Strategies Predicted by Optimal Feedback Control , 2007, The Journal of Neuroscience.

[68]  Robert E. Kass,et al.  2009 Special Issue: Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms , 2009 .

[69]  Ziv M. Williams,et al.  A cortical-spinal prosthesis for targeted limb movement in paralyzed primate avatars , 2014, Nature Communications.

[70]  David Sussillo,et al.  A recurrent neural network for closed-loop intracortical brain–machine interface decoders , 2012, Journal of neural engineering.

[71]  A B Schwartz,et al.  Motor cortical representation of speed and direction during reaching. , 1999, Journal of neurophysiology.

[72]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[73]  Sanggyun Kim,et al.  Team decision theory and brain-machine interfaces , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[74]  Rajesh P. N. Rao,et al.  Cortical activity during motor execution, motor imagery, and imagery-based online feedback , 2010, Proceedings of the National Academy of Sciences.

[75]  Ziv M. Williams,et al.  A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design , 2013, PloS one.

[76]  Justin C. Sanchez,et al.  A Symbiotic Brain-Machine Interface through Value-Based Decision Making , 2011, PloS one.

[77]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[78]  Jose M. Carmena,et al.  Continuous Closed-Loop Decoder Adaptation with a Recursive Maximum Likelihood Algorithm Allows for Rapid Performance Acquisition in Brain-Machine Interfaces , 2014, Neural Computation.

[79]  D. Moran,et al.  Cortical Adaptation to a Chronic Micro-Electrocorticographic Brain Computer Interface , 2013, The Journal of Neuroscience.

[80]  Miguel A. L. Nicolelis,et al.  Adaptive Decoding for Brain-Machine Interfaces Through Bayesian Parameter Updates , 2011, Neural Computation.

[81]  Robert E. Kass,et al.  A Spike-Train Probability Model , 2001, Neural Computation.

[82]  E. Fetz.  Operant Conditioning of Cortical Unit Activity , 1969, Science.

[83]  Nicholas G Hatsopoulos,et al.  Improving brain–machine interface performance by decoding intended future movements , 2013, Journal of neural engineering.

[84]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.

[85]  Rajesh P. N. Rao,et al.  Distributed cortical adaptation during learning of a brain–computer interface task , 2013, Proceedings of the National Academy of Sciences.

[86]  Robert E. Kass,et al.  Comparison of brain–computer interface decoding algorithms in open-loop and closed-loop control , 2010, Journal of Computational Neuroscience.