Decoding Neural Signals with a Compact and Interpretable Convolutional Neural Network

Brain-computer interfaces (BCIs) decode information from neural activity and send it to external devices. In recent years, we have seen an emergence of new algorithms for BCI decoding. Here we propose a compact architecture for adaptive decoding of electrocorticographic (ECoG) data into finger kinematics. We also describe a theoretically justified approach to interpreting the spatial and temporal weights in the architectures that combine adaptation in both space and time, such as ours. In hese architectures the weights are optimized not only for decoding of target signals but also for tuning away from the interfering sources, in both the spatial and the frequency domains. When applied to a dataset taken from the repository of Berlin BCI IV competition, our architecture outperformed the competition winners without the need for feature selection. Moreover, by looking at the architecture weights we could explain in physiological terms how our algorithm decodes spatial and temporal parameters of finger kinematics. As such, the proposed architecture offers a good decoder and a tool for investigating neural mechanisms of motor control.

[1]  Laurent Bougrain,et al.  Decoding Finger Flexion from Band-Specific ECoG Signals in Humans , 2012, Front. Neurosci..

[2]  C. Moore,et al.  The rate of transient beta frequency events predicts behavior across tasks and species , 2017, eLife.

[3]  Eran Dayan,et al.  Alpha and Beta Band Event-Related Desynchronization Reflects Kinematic Regularities , 2015, The Journal of Neuroscience.

[4]  Hanna-Leena Halme,et al.  Adaptive neural network classifier for decoding MEG signals , 2018, NeuroImage.

[5]  Nicholas G Hatsopoulos,et al.  The science of neural interface systems. , 2009, Annual review of neuroscience.

[6]  M. Nicolelis,et al.  Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. , 2017, Physiological reviews.

[7]  N. Birbaumer,et al.  Brain–computer interfaces for communication and rehabilitation , 2016, Nature Reviews Neurology.

[8]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[9]  Jing Wang,et al.  A Brain-to-Brain Interface for Real-Time Sharing of Sensorimotor Information , 2013, Scientific Reports.

[10]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[11]  M. Hines,et al.  Human Neocortical Neurosolver (HNN), a new software tool for interpreting the cellular and network origin of human MEG/EEG data , 2020, eLife.

[12]  Tiago H. Falk,et al.  Deep learning-based electroencephalography analysis: a systematic review , 2019, Journal of neural engineering.

[13]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[14]  Yoshua Bengio,et al.  Speaker Recognition from Raw Waveform with SincNet , 2018, 2018 IEEE Spoken Language Technology Workshop (SLT).

[15]  Klaus-Robert Müller,et al.  Introduction to machine learning for brain imaging , 2011, NeuroImage.

[16]  Ayman Atia,et al.  Brain computer interfacing: Applications and challenges , 2015 .

[17]  G. Buzsáki Rhythms of the brain , 2006 .

[18]  Jaime Gómez Gil,et al.  Brain Computer Interfaces, a Review , 2012, Sensors.

[19]  Stefan L. Hahn,et al.  On the uniqueness of the definition of the amplitude and phase of the analytic signal , 2003, Signal Process..

[20]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human EEG , 2017, ArXiv.

[21]  O. Arias-Carrión,et al.  EEG-based Brain-Computer Interfaces: An Overview of Basic Concepts and Clinical Applications in Neurorehabilitation , 2010, Reviews in the neurosciences.

[22]  G. Schalk,et al.  Brain-Computer Interfaces Using Electrocorticographic Signals , 2011, IEEE Reviews in Biomedical Engineering.

[23]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[24]  J. Wolpaw,et al.  Clinical Applications of Brain-Computer Interfaces: Current State and Future Prospects , 2009, IEEE Reviews in Biomedical Engineering.

[25]  Mikhail A. Lebedev,et al.  Decoding Movement From Electrocorticographic Activity: A Review , 2019, Front. Neuroinform..

[26]  Mark L. Homer,et al.  Sensors and decoding for intracortical brain computer interfaces. , 2013, Annual review of biomedical engineering.

[27]  C. Neuper,et al.  Resting-state sensorimotor rhythm (SMR) power predicts the ability to up-regulate SMR in an EEG-instrumental conditioning paradigm , 2015, Clinical Neurophysiology.