Information Systems Opportunities in Brain-Machine Interface

Brain-machine interface (BMI) systems convert neural signals from motor regions of the brain into control signals to guide prosthetic devices. The ultimate goal of BMIs is to improve the quality of life for people with paralysis by providing direct neural control of prosthetic arms or computer cursors. While considerable research over the past 15 years has led to compelling BMI demonstrations, there remain several challenges to achieving clinically viable BMI systems. In this review, we focus on the challenge of increasing BMI perfor- mance and robustness. We review and highlight key aspects of intracortical BMI decoder design, which is central to the conversion of neural signals into prosthetic control signals, and discuss emerging opportunities to improve intracortical BMI decoders. This is one of the primary research opportunities where information systems engineering can directly impact the future success of BMIs.

[1]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[2]  G. Schalk,et al.  Evolution of brain-computer interfaces : going beyond classic motor physiology , 2009 .

[3]  Deniz Erdogmus,et al.  Interpreting neural activity through linear and nonlinear models for brain machine interfaces , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[4]  P R Kennedy,et al.  Direct control of a computer from the human central nervous system. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[5]  Michael J. Black,et al.  A quantitative comparison of linear and non-linear models of motor cortical activity for the encoding and decoding of arm motions , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[6]  Deniz Erdogmus,et al.  Learning mappings in brain machine interfaces with echo state networks , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[7]  Arjun K. Bansal,et al.  Relationships among low-frequency local field potentials, spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. , 2011, Journal of neurophysiology.

[8]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[9]  R. Andersen,et al.  Cortical Local Field Potential Encodes Movement Intentions in the Posterior Parietal Cortex , 2005, Neuron.

[10]  K. Shenoy,et al.  Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex. , 2007, Journal of neurophysiology.

[11]  Byron M. Yu,et al.  A high-performance brain–computer interface , 2006, Nature.

[12]  Teresa H. Y. Meng,et al.  HermesD: A High-Rate Long-Range Wireless Transmission System for Simultaneous Multichannel Neural Recording Applications , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[13]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[14]  Bijan Pesaran,et al.  Optimizing the Decoding of Movement Goals from Local Field Potentials in Macaque Cortex , 2011, The Journal of Neuroscience.

[15]  Maneesh Sahani,et al.  Learning stable, regularised latent models of neural population dynamics , 2012, Network.

[16]  U. Mitzdorf Current source-density method and application in cat cerebral cortex: investigation of evoked potentials and EEG phenomena. , 1985, Physiological reviews.

[17]  Gopal Santhanam,et al.  Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. , 2006, Journal of neurophysiology.

[18]  Robert D Flint,et al.  Local field potentials allow accurate decoding of muscle activity. , 2012, Journal of neurophysiology.

[19]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[20]  Robert E Kass,et al.  Statistical issues in the analysis of neuronal data. , 2005, Journal of neurophysiology.

[21]  Vikash Gilja,et al.  A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces. , 2011, Journal of neurophysiology.

[22]  L. Miller,et al.  Accurate decoding of reaching movements from field potentials in the absence of spikes , 2012, Journal of neural engineering.

[23]  H. Sorenson Least-squares estimation: from Gauss to Kalman , 1970, IEEE Spectrum.

[24]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[25]  Arto Nurmikko,et al.  An implantable wireless neural interface for recording cortical circuit dynamics in moving primates , 2013, Journal of neural engineering.

[26]  John P. Cunningham,et al.  Dynamical segmentation of single trials from population neural data , 2011, NIPS.

[27]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[28]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[29]  R E Kass,et al.  Recursive bayesian decoding of motor cortical signals by particle filtering. , 2004, Journal of neurophysiology.

[30]  David Sussillo,et al.  A recurrent neural network for closed-loop intracortical brain–machine interface decoders , 2012, Journal of neural engineering.

[31]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[32]  Krishna V. Shenoy,et al.  Information Systems Opportunities in Brain–Machine Interface Decoders , 2014, Proceedings of the IEEE.

[33]  Kathie L. Olsen,et al.  Neurotech for Neuroscience: Unifying Concepts, Organizing Principles, and Emerging Tools , 2007, The Journal of Neuroscience.

[34]  Eilon Vaadia,et al.  Kernel-ARMA for Hand Tracking and Brain-Machine interfacing During 3D Motor Control , 2008, NIPS.

[35]  Ming Yin,et al.  Listening to Brain Microcircuits for Interfacing With External World—Progress in Wireless Implantable Microelectronic Neuroengineering Devices , 2010, Proceedings of the IEEE.

[36]  Justin C. Sanchez,et al.  Comprehensive characterization and failure modes of tungsten microwire arrays in chronic neural implants , 2012, Journal of neural engineering.

[37]  C. Mehring,et al.  Inference of hand movements from local field potentials in monkey motor cortex , 2003, Nature Neuroscience.

[38]  C. Koch,et al.  The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes , 2012, Nature Reviews Neuroscience.

[39]  K. Anderson Targeting recovery: priorities of the spinal cord-injured population. , 2004, Journal of neurotrauma.

[40]  E. Fetz Operant Conditioning of Cortical Unit Activity , 1969, Science.

[41]  Eran Stark,et al.  Predicting Movement from Multiunit Activity , 2007, The Journal of Neuroscience.

[42]  Qiang Ji,et al.  Decoding onset and direction of movements using Electrocorticographic (ECoG) signals in humans , 2012, Front. Neuroeng..

[43]  Stuart K. Card,et al.  Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys, for text selection on a CRT , 1987 .

[44]  Emilio Salinas,et al.  Vector reconstruction from firing rates , 1994, Journal of Computational Neuroscience.

[45]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[46]  Mark L. Homer,et al.  Sensors and decoding for intracortical brain computer interfaces. , 2013, Annual review of biomedical engineering.

[47]  D.A. Heldman,et al.  Local field potential spectral tuning in motor cortex during reaching , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[48]  Michael J. Black,et al.  Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array , 2011 .

[49]  J. M. Carmena,et al.  Closed-Loop Decoder Adaptation on Intermediate Time-Scales Facilitates Rapid BMI Performance Improvements Independent of Decoder Initialization Conditions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[50]  Eun Jung Hwang,et al.  The utility of multichannel local field potentials for brain–machine interfaces , 2013, Journal of neural engineering.

[51]  R.R. Harrison,et al.  HermesC: Low-Power Wireless Neural Recording System for Freely Moving Primates , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[52]  Stefano Panzeri,et al.  Modelling and analysis of local field potentials for studying the function of cortical circuits , 2013, Nature Reviews Neuroscience.

[53]  Maneesh Sahani,et al.  A dynamical systems view of motor preparation: implications for neural prosthetic system design. , 2011, Progress in brain research.

[54]  Uri T Eden,et al.  A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. , 2005, Journal of neurophysiology.

[55]  Stephen I. Ryu,et al.  Neural Dynamics of Reaching following Incorrect or Absent Motor Preparation , 2014, Neuron.

[56]  Sherwin S Chan,et al.  Motor cortical representation of position and velocity during reaching. , 2007, Journal of neurophysiology.

[57]  Saeed Vaseghi Least Square Error Wiener‐Kolmogorov Filters , 2007 .

[58]  Robert E. Kass,et al.  2009 Special Issue: Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms , 2009 .

[59]  C. Mehring,et al.  Encoding of Movement Direction in Different Frequency Ranges of Motor Cortical Local Field Potentials , 2005, The Journal of Neuroscience.

[60]  Jiping He,et al.  Selection and parameterization of cortical neurons for neuroprosthetic control , 2006, Journal of neural engineering.

[61]  Thomas Kailath,et al.  A view of three decades of linear filtering theory , 1974, IEEE Trans. Inf. Theory.

[62]  Dragan F. Dimitrov,et al.  Reversible large-scale modification of cortical networks during neuroprosthetic control , 2011, Nature Neuroscience.

[63]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[64]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[65]  Paul Nuyujukian,et al.  Intention estimation in brain–machine interfaces , 2014, Journal of neural engineering.

[66]  Kwabena Boahen,et al.  Design and validation of a real-time spiking-neural-network decoder for brain–machine interfaces , 2013, Journal of neural engineering.

[67]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications , 1949 .

[68]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[69]  P. Kennedy,et al.  Restoration of neural output from a paralyzed patient by a direct brain connection , 1998, Neuroreport.

[70]  Nicolas Y. Masse,et al.  Advantages of closed-loop calibration in intracortical brain–computer interfaces for people with tetraplegia , 2013, Journal of neural engineering.

[71]  Robert D Flint,et al.  Long term, stable brain machine interface performance using local field potentials and multiunit spikes , 2013, Journal of neural engineering.

[72]  Kelvin So,et al.  Subject-specific modulation of local field potential spectral power during brain–machine interface control in primates , 2014, Journal of neural engineering.

[73]  J. Donoghue,et al.  Failure mode analysis of silicon-based intracortical microelectrode arrays in non-human primates , 2013, Journal of neural engineering.

[74]  Joseph E. O’Doherty,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[75]  L R Hochberg,et al.  Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[76]  M. Sahani,et al.  Cortical control of arm movements: a dynamical systems perspective. , 2013, Annual review of neuroscience.

[77]  Nicholas Hatsopoulos,et al.  Decoding continuous and discrete motor behaviors using motor and premotor cortical ensembles. , 2004, Journal of neurophysiology.

[78]  Hannes Bleuler,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[79]  J. Donoghue,et al.  Primary Motor Cortex Tuning to Intended Movement Kinematics in Humans with Tetraplegia , 2008, The Journal of Neuroscience.

[80]  Arjun K. Bansal,et al.  Decoding 3D reach and grasp from hybrid signals in motor and premotor cortices: spikes, multiunit activity, and local field potentials. , 2012, Journal of neurophysiology.

[81]  G. Rizzolatti,et al.  Seven Years of Recording from Monkey Cortex with a Chronically Implanted Multiple Microelectrode , 2010, Front. Neuroeng..

[82]  J. Kalaska,et al.  Learning to Move Machines with the Mind , 2022 .

[83]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[84]  J.C. Sanchez,et al.  Simultaneus prediction of four kinematic variables for a brain-machine interface using a single recurrent neural network , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[85]  R. Normann,et al.  Thermal Impact of an Active 3-D Microelectrode Array Implanted in the Brain , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[86]  Kwabena Boahen,et al.  A Brain-Machine Interface Operating with a Real-Time Spiking Neural Network Control Algorithm , 2011, NIPS.

[87]  Steven M Chase,et al.  Control of a brain–computer interface without spike sorting , 2009, Journal of neural engineering.

[88]  Matthew Fellows,et al.  Statistical encoding model for a primary motor cortical brain-machine interface , 2005, IEEE Transactions on Biomedical Engineering.

[89]  M. Carandini,et al.  Local Origin of Field Potentials in Visual Cortex , 2009, Neuron.

[90]  Krishna V. Shenoy,et al.  Challenges and Opportunities for Next-Generation Intracortically Based Neural Prostheses , 2011, IEEE Transactions on Biomedical Engineering.

[91]  Dragan F. Dimitrov,et al.  Cortical Representation of Ipsilateral Arm Movements in Monkey and Man , 2009, The Journal of Neuroscience.

[92]  Byron M. Yu,et al.  Single-Trial Neural Correlates of Arm Movement Preparation , 2011, Neuron.

[93]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[94]  J. Wolpaw,et al.  Decoding two-dimensional movement trajectories using electrocorticographic signals in humans , 2007, Journal of neural engineering.

[95]  R. Kass,et al.  Approximate Methods for State-Space Models , 2010, Journal of the American Statistical Association.

[96]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[97]  John P. Donoghue,et al.  Decoding 3-D Reach and Grasp Kinematics From High-Frequency Local Field Potentials in Primate Primary Motor Cortex , 2010, IEEE Transactions on Biomedical Engineering.

[98]  P. Wolf Thermal Considerations for the Design of an Implanted Cortical Brain–Machine Interface (BMI) , 2008 .

[99]  Krishna V. Shenoy,et al.  Monkey models for brain-machine interfaces: The need for maintaining diversity , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[100]  Wei Wu,et al.  Neural Decoding of Cursor Motion Using a Kalman Filter , 2002, NIPS.

[101]  Robert E Kass,et al.  Functional network reorganization during learning in a brain-computer interface paradigm , 2008, Proceedings of the National Academy of Sciences.

[102]  Vikash Gilja,et al.  Long-term Stability of Neural Prosthetic Control Signals from Silicon Cortical Arrays in Rhesus Macaque Motor Cortex , 2010 .

[103]  Robert E. Kass,et al.  Comparison of brain–computer interface decoding algorithms in open-loop and closed-loop control , 2010, Journal of Computational Neuroscience.

[104]  Michael J. Black,et al.  Probabilistic Inference of Hand Motion from Neural Activity in Motor Cortex , 2001, NIPS.

[105]  Daniel Moran,et al.  Evolution of brain–computer interface: action potentials, local field potentials and electrocorticograms , 2010, Current Opinion in Neurobiology.

[106]  F. Solzbacher,et al.  Integrated wireless neural interface based on the Utah electrode array , 2009, Biomedical microdevices.