FingerFlex: Inferring Finger Trajectories from ECoG signals

Motor brain-computer interface (BCI) development relies critically on neural time series decoding algorithms. Recent advances in deep learning architectures allow for automatic feature selection to approximate higher-order dependencies in data. This article presents the FingerFlex model - a convolutional encoder-decoder architecture adapted for finger movement regression on electrocorticographic (ECoG) brain data. State-of-the-art performance was achieved on a publicly available BCI competition IV dataset 4 with a correlation coefficient between true and predicted trajectories up to 0.74. The presented method provides the opportunity for developing fully-functional high-precision cortical motor brain-computer interfaces.

[1]  Bingzhao Zhu,et al.  Fast and accurate decoding of finger movements from ECoG through Riemannian features and modern machine learning techniques , 2022, Journal of neural engineering.

[2]  Antoine Souloumiac,et al.  Decoding ECoG signal into 3D hand translation using deep learning , 2021, Journal of neural engineering.

[3]  Jingyu Liu,et al.  A Temporal-Spectral-Based Squeeze-and- Excitation Feature Fusion Network for Motor Imagery EEG Decoding , 2021, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Guangming Shi,et al.  Decoding imagined speech from EEG signals using hybrid-scale spatial-temporal dilated convolution network , 2021, Journal of neural engineering.

[5]  M. Lebedev,et al.  Decoding and interpreting cortical signals with a compact convolutional neural network , 2021, Journal of neural engineering.

[6]  Xiaohui Liu,et al.  A review on transfer learning in EEG signal analysis , 2021, Neurocomputing.

[7]  Rajesh P. N. Rao,et al.  Generalized neural decoders for transfer learning across participants and recording modalities , 2020, bioRxiv.

[8]  C Perrodin,et al.  Interpreting wide-band neural activity using convolutional neural networks , 2019, bioRxiv.

[9]  Mikhail A. Lebedev,et al.  Decoding Movement From Electrocorticographic Activity: A Review , 2019, Front. Neuroinform..

[10]  Mahsa Shoaran,et al.  Enhanced Classification of Individual Finger Movements with ECoG , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[11]  E. Keefer,et al.  Human motor decoding from neural signals: a review , 2019, BMC biomedical engineering.

[12]  K. Miller.  A library of human electrocorticographic data and analyses , 2019, Nature Human Behaviour.

[13]  Marie-Caroline Schaeffer,et al.  Decoding of finger activation from ECoG data: a comparative study , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[14]  Ronan Collobert,et al.  wav2vec: Unsupervised Pre-training for Speech Recognition , 2019, INTERSPEECH.

[15]  Gang Pan,et al.  Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks , 2018, Front. Neurosci..

[16]  Odelia Schwartz,et al.  Decoding of finger trajectory from ECoG using deep learning , 2018, Journal of neural engineering.

[17]  Francisco J. Valero Cuevas,et al.  Finger movements are mainly represented by a linear transformation of energy in band-specific ECoG signals , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[18]  Alexandre Barachant,et al.  Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review , 2017 .

[19]  Nick F. Ramsey,et al.  Decoding hand gestures from primary somatosensory cortex using high-density ECoG , 2017, NeuroImage.

[20]  Manal Abdel Wahed,et al.  Decoding of finger movement using kinematic model classification and regression model switching , 2016, 2016 8th Cairo International Biomedical Engineering Conference (CIBEC).

[21]  Klaus-Robert Müller,et al.  Interpretable deep neural networks for single-trial EEG classification , 2016, Journal of Neuroscience Methods.

[22]  Nicholas P. Szrama,et al.  Decoding three-dimensional reaching movements using electrocorticographic signals in humans , 2016, Journal of neural engineering.

[23]  Kapil D. Katyal,et al.  Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject , 2016, Journal of neural engineering.

[24]  Apostolos P Georgopoulos,et al.  Coding of movements in the motor cortex , 2015, Current Opinion in Neurobiology.

[25]  Mahdi Bamdad,et al.  Application of BCI systems in neurorehabilitation: a scoping review , 2015, Disability and rehabilitation. Assistive technology.

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  Donald J Bolger,et al.  The neurophysiological bases of EEG and EEG measurement: a review for the rest of us. , 2014, Psychophysiology.

[28]  Yasuharu Koike,et al.  Decoding fingertip trajectory from electrocorticographic signals in humans , 2014, Neuroscience Research.

[29]  Bin He,et al.  Brain–Computer Interfaces Using Sensorimotor Rhythms: Current State and Future Perspectives , 2014, IEEE Transactions on Biomedical Engineering.

[30]  Yasuharu Koike,et al.  Prediction of Three-Dimensional Arm Trajectories Based on ECoG Signals Recorded from Human Sensorimotor Cortex , 2013, PloS one.

[31]  Laurent Bougrain,et al.  Decoding Finger Flexion from Band-Specific ECoG Signals in Humans , 2012, Front. Neurosci..

[32]  C. Koch,et al.  The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes , 2012, Nature Reviews Neuroscience.

[33]  Alain Rakotomamonjy,et al.  Decoding Finger Movements from ECoG Signals Using Switching Linear Models , 2011, Front. Neurosci..

[34]  Andreas Schulze-Bonhage,et al.  Prediction of arm movement trajectories from ECoG-recordings in humans , 2008, Journal of Neuroscience Methods.

[35]  J. Wolpaw,et al.  Decoding two-dimensional movement trajectories using electrocorticographic signals in humans , 2007, Journal of neural engineering.

[36]  Miguel A. L. Nicolelis,et al.  Brain–machine interfaces: past, present and future , 2006, Trends in Neurosciences.

[37]  G. Buzsáki,et al.  Neuronal Oscillations in Cortical Networks , 2004, Science.

[38]  Kung Yao,et al.  Source localization and beamforming , 2002, IEEE Signal Process. Mag..

[39]  M. Schieber.  Constraints on somatotopic organization in the primary motor cortex. , 2001, Journal of neurophysiology.

[40]  D. Hoffman,et al.  Muscle and movement representations in the primary motor cortex. , 1999, Science.

[41]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[42]  R. Ilmoniemi,et al.  Magnetoencephalography-theory, instrumentation, and applications to noninvasive studies of the working human brain , 1993 .

[43]  D. Cohen,et al.  Comparison of the magnetoencephalogram and electroencephalogram. , 1979, Electroencephalography and clinical neurophysiology.

[44]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[45]  Ibrahim Onaran,et al.  Classification of multichannel ECoG related to individual finger movements with redundant spatial projections , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[46]  A. Engel,et al.  Invasive recordings from the human brain: clinical insights and beyond , 2005, Nature Reviews Neuroscience.