Encoder-Decoder Optimization for Brain-Computer Interfaces

Neuroprosthetic brain-computer interfaces are systems that decode neural activity into useful control signals for effectors, such as a cursor on a computer screen. It has long been recognized that both the user and decoding system can adapt to increase the accuracy of the end effector. Co-adaptation is the process whereby a user learns to control the system in conjunction with the decoder adapting to learn the user's neural patterns. We provide a mathematical framework for co-adaptation and relate co-adaptation to the joint optimization of the user's control scheme ("encoding model") and the decoding algorithm's parameters. When the assumptions of that framework are respected, co-adaptation cannot yield better performance than that obtainable by an optimal initial choice of fixed decoder, coupled with optimal user learning. For a specific case, we provide numerical methods to obtain such an optimized decoder. We demonstrate our approach in a model brain-computer interface system using an online prosthesis simulator, a simple human-in-the-loop pyschophysics setup which provides a non-invasive simulation of the BCI setting. These experiments support two claims: that users can learn encoders matched to fixed, optimal decoders and that, once learned, our approach yields expected performance advantages.

[1]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[2]  M. Casadio,et al.  Body machine interface: Remapping motor skills after spinal cord injury , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[3]  L R Hochberg,et al.  Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Byron M. Yu,et al.  Motor cortical control of movement speed with implications for brain-machine interface control. , 2014, Journal of neurophysiology.

[5]  Kyong-Hwa Lee,et al.  Optimal Linear Coding for Vector Channels , 1976, IEEE Trans. Commun..

[6]  Ferdinando A Mussa-Ivaldi,et al.  Remapping hand movements in a novel geometrical environment. , 2005, Journal of neurophysiology.

[7]  W. Murray,et al.  An algorithm for RLS identification parameters that vary quickly with time , 1993, IEEE Trans. Autom. Control..

[8]  Robert E. Kass,et al.  Comparison of brain–computer interface decoding algorithms in open-loop and closed-loop control , 2010, Journal of Computational Neuroscience.

[9]  Daniel M. Wolpert,et al.  Making smooth moves , 2022 .

[10]  R. Andersen,et al.  Volitional Control of Neural Activity Relies on the Natural Motor Repertoire , 2013, Current Biology.

[11]  Michael S. Lewicki,et al.  Characterization of Minimum Error Linear Coding with Sensory and Neural Noise , 2011, Neural Computation.

[12]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[13]  Alan J. Laub,et al.  Matrix analysis - for scientists and engineers , 2004 .

[14]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[15]  Roy Fox,et al.  A multi-agent control framework for co-adaptation in brain-computer interfaces , 2013, NIPS.

[16]  Zachary Danziger,et al.  Learning Algorithms for Human–Machine Interfaces , 2009, IEEE Transactions on Biomedical Engineering.

[17]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[18]  Michael S. Lewicki,et al.  Robust Coding Over Noisy Overcomplete Channels , 2007, IEEE Transactions on Image Processing.

[19]  Jose M. Carmena,et al.  Closed-Loop Decoder Adaptation Shapes Neural Plasticity for Skillful Neuroprosthetic Control , 2014, Neuron.

[20]  Byron M. Yu,et al.  Neural constraints on learning , 2014, Nature.

[21]  E. E. Fetz,et al.  Interfacing With the Computational Brain , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[23]  D. Simon Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches , 2006 .

[24]  John P. Cunningham,et al.  Empirical models of spiking in neural populations , 2011, NIPS.

[25]  Eero P. Simoncelli,et al.  Efficient Coding of Spatial Information in the Primate Retina , 2012, The Journal of Neuroscience.

[26]  Byron M. Yu,et al.  Self-recalibrating classifiers for intracortical brain–computer interfaces , 2014, Journal of neural engineering.

[27]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[28]  Bryan D. He,et al.  Neural shaping with joint optimization of controller and plant under restricted dynamics , 2014, 2014 Information Theory and Applications Workshop (ITA).

[29]  Robert E Kass,et al.  Functional network reorganization during learning in a brain-computer interface paradigm , 2008, Proceedings of the National Academy of Sciences.

[30]  Vikash Gilja,et al.  A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces. , 2011, Journal of neurophysiology.

[31]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[32]  Wei Wu,et al.  Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter , 2006, Neural Computation.

[33]  Bryan D. He,et al.  Dynamic Analysis of Naive Adaptive Brain-Machine Interfaces , 2013, Neural Computation.

[34]  Rajiv Ranganathan,et al.  The Body-Machine Interface: A New Perspective on an Old Theme , 2012, Journal of motor behavior.

[35]  A. Schwartz,et al.  Behavioral and neural correlates of visuomotor adaptation observed through a brain-computer interface in primary motor cortex. , 2012, Journal of neurophysiology.

[36]  Zoubin Ghahramani,et al.  Unifying linear dimensionality reduction , 2014, 1406.0873.

[37]  Nicholas P. Szrama,et al.  Using the electrocorticographic speech network to control a brain–computer interface in humans , 2011, Journal of neural engineering.

[38]  Dragan F. Dimitrov,et al.  Reversible large-scale modification of cortical networks during neuroprosthetic control , 2011, Nature Neuroscience.

[39]  Lakshminarayan Srinivasan,et al.  Stochastic Optimal Control as a Theory of Brain-Machine Interface Operation , 2013, Neural Computation.

[40]  José Carlos Príncipe,et al.  Coadaptive Brain–Machine Interface via Reinforcement Learning , 2009, IEEE Transactions on Biomedical Engineering.

[41]  J. Kalaska,et al.  Learning to Move Machines with the Mind , 2022 .

[42]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[43]  Y. Ho,et al.  A Bayesian approach to problems in stochastic estimation and control , 1964 .

[44]  Zachary Danziger A reductionist approach to the analysis of learning in brain–computer interfaces , 2014, Biological Cybernetics.

[45]  Jose M. Carmena,et al.  Continuous Closed-Loop Decoder Adaptation with a Recursive Maximum Likelihood Algorithm Allows for Rapid Performance Acquisition in Brain-Machine Interfaces , 2014, Neural Computation.

[46]  John P. Cunningham,et al.  A brain machine interface control algorithm designed from a feedback control perspective , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[47]  D. Sinha,et al.  On the selection of measurements in least-squares estimation , 1989, IEEE 1989 International Conference on Systems Engineering.

[48]  Krishna V. Shenoy,et al.  Combining Decoder Design and Neural Adaptation in Brain-Machine Interfaces , 2014, Neuron.

[49]  Miguel A. L. Nicolelis,et al.  Adaptive Decoding for Brain-Machine Interfaces Through Bayesian Parameter Updates , 2011, Neural Computation.