Near Perfect Neural Critic from Motor Cortical Activity Toward an Autonomously Updating Brain Machine Interface

We are developing an autonomously updating brain machine interface (BMI) utilizing reinforcement learning principles. One aspect of this system is a neural critic that determines reward expectations from neural activity. This critic is then used to update a BMI decoder toward an improved performance from the user's perspective. Here we demonstrate the ability of a neural critic to classify trial reward value given activity from the primary motor cortex (M1), using neural features from single/multi units (SU/MU), and local field potentials (LFPs) with prediction accuracies up to 97% correct. A nonhuman primate subject conducted a cued center out reaching task, either manually, or observationally. The cue indicated the reward value of a trial. Features such as power spectral density (PSD) of the LFPs and spike-field coherence (SFC) between SU/MU and corresponding LFPs were calculated and used as inputs to several classifiers. We conclude that hybrid features of PSD and SFC show higher classification performance than PSD or SFC alone (accuracy was 92% for manual tasks, and 97% for observational). In the future, we will employ these hybrid features toward our autonomously updating BMI.

[1]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Joseph T. Francis,et al.  Toward an Autonomous Brain Machine Interface: Integrating Sensorimotor Reward Modulation and Reinforcement Learning , 2015, The Journal of Neuroscience.

[3]  José Carlos Príncipe,et al.  Coadaptive Brain–Machine Interface via Reinforcement Learning , 2009, IEEE Transactions on Biomedical Engineering.

[4]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[5]  T. Hafting,et al.  Frequency of gamma oscillations routes flow of information in the hippocampus , 2009, Nature.

[6]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[7]  R. Desimone,et al.  Modulation of Oscillatory Neuronal Synchronization by Selective Visual Attention , 2001, Science.

[8]  Joseph T. Francis,et al.  Reward value is encoded in primary somatosensory cortex and can be decoded from neural activity during performance of a psychophysical task , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[9]  W. Singer,et al.  Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Justin C. Sanchez,et al.  A confidence metric for using neurobiological feedback in actor-critic reinforcement learning based brain-machine interfaces , 2014, Front. Neurosci..

[11]  Aurélien Géron,et al.  Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems , 2017 .

[12]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[13]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[14]  Partha P. Mitra,et al.  Sampling Properties of the Spectrum and Coherency of Sequences of Action Potentials , 2000, Neural Computation.

[15]  Aaron C. Koralek,et al.  Temporally Precise Cell-Specific Coherence Develops in Corticostriatal Networks during Learning , 2013, Neuron.

[16]  José Carlos Príncipe,et al.  Exploiting co-adaptation for the design of symbiotic neuroprosthetic assistants , 2009, Neural Networks.

[17]  Joseph T. Francis,et al.  Properties of a temporal difference reinforcement learning brain machine interface driven by a simulated motor cortex , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .