Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain–computer interfaces

Brain-computer interfaces (BCIs) aim to restore independence to people with severe motor disabilities by allowing control of acursor on a computer screen or other effectors with neural activity. However, physiological and/or recording-related nonstationarities in neural signals can limit long-term decoding stability, and it would be tedious for users to pause use of the BCI whenever neural control degrades to perform decoder recalibration routines. We recently demonstrated that a kinematic decoder (i.e. a decoder that controls cursor movement) can be recalibrated using data acquired during practical point-and-click control of the BCI by retrospectively inferring users' intended movement directions based on their subsequent selections. Here, we extend these methods to allow the click decoder to also be recalibrated using data acquired during practical BCI use. We retrospectively labeled neural data patterns as corresponding to "click" during all time bins in which the click log-likelihood (decoded using linear discriminant analysis, or LDA) had been above the click threshold that was used during real-time neural control. We labeled as "non-click" those periods that the kinematic decoder's retrospective target inference (RTI) heuristics determined to be consistent with intended cursor movement. Once these neural activity patterns were labeled, the click decoder was calibrated using standard supervised classifier training methods. Combined with real-time bias correction and baseline firing rate tracking, this set of "retrospectively labeled" decoder calibration methods enabled a BrainGate participant with amyotrophic lateral sclerosis (T9) to type freely across 11 research sessions spanning 29days, maintaining high-performance neural control over cursor movement and click without needing to interrupt virtual keyboard use for explicit calibration tasks. By eliminating the need for tedious calibration tasks with prescribed targets and pre-specified click times, this approach advances the potential clinical utility of intracortical BCIs for individuals with severe motor disability.

[1]  Ricardo Chavarriaga,et al.  Errare machinale est: the use of error-related potentials in brain-machine interfaces , 2014, Front. Neurosci..

[2]  Teresa H. Y. Meng,et al.  HermesB: A Continuous Neural Recording System for Freely Behaving Primates , 2007, IEEE Transactions on Biomedical Engineering.

[3]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[4]  J. Blumberg,et al.  Adaptive Classification for Brain Computer Interfaces , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[5]  Byron M. Yu,et al.  Factor-analysis methods for higher-performance neural prostheses. , 2009, Journal of neurophysiology.

[6]  M. Nicolelis,et al.  Optimizing a Linear Algorithm for Real-Time Robotic Control using Chronic Cortical Ensemble Recordings in Monkeys , 2004, Journal of Cognitive Neuroscience.

[7]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[8]  Michael J. Black,et al.  Multi-state decoding of point-and-click control signals from motor cortical activity in a human with tetraplegia , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[9]  L R Hochberg,et al.  Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[10]  Rajesh P. N. Rao,et al.  Towards adaptive classification for BCI , 2006, Journal of neural engineering.

[11]  Me Meluin Bruce,et al.  Estimation of variance by a recursive equation , 1969 .

[12]  M L Boninger,et al.  Ten-dimensional anthropomorphic arm control in a human brain−machine interface: difficulties, solutions, and limitations , 2015, Journal of neural engineering.

[13]  Nicolas Y. Masse,et al.  Neural Point-and-Click Communication by a Person With Incomplete Locked-In Syndrome , 2015, Neurorehabilitation and neural repair.

[14]  Steven M Chase,et al.  Control of a brain–computer interface without spike sorting , 2009, Journal of neural engineering.

[15]  Vikash Gilja,et al.  Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain–machine interface performance , 2015, Journal of neural engineering.

[16]  Paul Nuyujukian,et al.  Performance sustaining intracortical neural prostheses , 2014, Journal of neural engineering.

[17]  Alan Julian Izenman,et al.  Modern Multivariate Statistical Techniques , 2008 .

[18]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[19]  Philipp Nadel A Choice Of Kiplings Verse , 2016 .

[20]  Vikash Gilja,et al.  Long-term Stability of Neural Prosthetic Control Signals from Silicon Cortical Arrays in Rhesus Macaque Motor Cortex , 2010 .

[21]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[22]  Nicolas Y. Masse,et al.  Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface , 2015, Science Translational Medicine.

[23]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[24]  Michael J. Black,et al.  Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array , 2011 .

[25]  H. Alkadhi,et al.  Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. , 1997, Brain : a journal of neurology.

[26]  Gerhard Friehs,et al.  Intra-day signal instabilities affect decoding performance in an intracortical neural interface system , 2013, Journal of neural engineering.

[27]  Nicolas Y. Masse,et al.  Non-causal spike filtering improves decoding of movement intention for intracortical BCIs , 2014, Journal of Neuroscience Methods.

[28]  Michael J. Black,et al.  Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[30]  Wei Wu,et al.  Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter , 2006, Neural Computation.

[31]  Francis R. Willett,et al.  High performance communication by people with paralysis using an intracortical brain-computer interface , 2017, eLife.

[32]  Byron M. Yu,et al.  Self-recalibrating classifiers for intracortical brain–computer interfaces , 2014, Journal of neural engineering.

[33]  Emery N. Brown,et al.  Modulation Depth Estimation and Variable Selection in State-Space Models for Neural Interfaces , 2015, IEEE Transactions on Biomedical Engineering.

[34]  Kip A Ludwig,et al.  Using a common average reference to improve cortical neuron recordings from microelectrode arrays. , 2009, Journal of neurophysiology.

[35]  Michael J. Black,et al.  Statistical Analysis of the Non-stationarity of Neural Population Codes , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[36]  Nicolas Y. Masse,et al.  Advantages of closed-loop calibration in intracortical brain–computer interfaces for people with tetraplegia , 2013, Journal of neural engineering.

[37]  Francis R. Willett,et al.  Restoration of reaching and grasping in a person with tetraplegia through brain-controlled muscle stimulation: a proof-of-concept demonstration , 2017, The Lancet.