Decoding kinetic features of hand motor preparation from single‐trial EEG using convolutional neural networks

Building accurate movement decoding models from brain signals is crucial for many biomedical applications. Predicting specific movement features, such as speed and force, before movement execution may provide additional useful information at the expense of increasing the complexity of the decoding problem. Recent attempts to predict movement speed and force from the electroencephalogram (EEG) achieved classification accuracies at or slightly above chance levels, highlighting the need for more accurate prediction strategies. Thus, the aims of this study were to accurately predict hand movement speed and force from single‐trial EEG signals and to decode neurophysiological information of motor preparation from the prediction strategies. To these ends, a decoding model based on convolutional neural networks (ConvNets) was implemented and compared against other state‐of‐the‐art prediction strategies, such as support vector machines and decision trees. ConvNets outperformed the other prediction strategies, achieving an overall accuracy of 84% in the classification of two different levels of speed and force (four‐class classification) from pre‐movement single‐trial EEG (100 ms and up to 1,600 ms prior to movement execution). Furthermore, an analysis of the ConvNet architectures suggests that the network performs a complex spatiotemporal integration of EEG data to optimize classification accuracy. These results show that movement speed and force can be accurately predicted from single‐trial EEG, and that the prediction strategies may provide useful neurophysiological information about motor preparation.

[1]  Tiago H. Falk,et al.  Deep learning-based electroencephalography analysis: a systematic review , 2019, Journal of neural engineering.

[2]  Bijan Pesaran,et al.  Differential roles of high gamma and local motor potentials for movement preparation and execution , 2016 .

[3]  Ke Liao,et al.  Decoding Individual Finger Movements from One Hand Using Human EEG Signals , 2014, PloS one.

[4]  Mads Jochumsen,et al.  A Review of Techniques for Detection of Movement Intention Using Movement-Related Cortical Potentials , 2015, Comput. Math. Methods Medicine.

[5]  D. Stuss,et al.  Effects of lateral lesions on medial activation during autobiographical remembering , 2010 .

[6]  Ricarda I. Schubotz,et al.  Prediction, Cognition and the Brain , 2009, Front. Hum. Neurosci..

[7]  Clemens Brunner,et al.  Better than random? A closer look on BCI results , 2008 .

[8]  M. Hallett,et al.  What is the Bereitschaftspotential? , 2006, Clinical Neurophysiology.

[9]  Marie-Françoise Lucas,et al.  Identification of task parameters from movement-related cortical potentials , 2009, Medical & Biological Engineering & Computing.

[10]  Vacius Jusas,et al.  Convolutional Neural Networks for Four-Class Motor Imagery Data Classification , 2017, IDC.

[11]  Trent J. Bradberry,et al.  Reconstructing Three-Dimensional Hand Movements from Noninvasive Electroencephalographic Signals , 2010, The Journal of Neuroscience.

[12]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[13]  Mads Jochumsen,et al.  Detection and classification of movement-related cortical potentials associated with task force and speed , 2013, Journal of neural engineering.

[14]  G Pfurtscheller,et al.  Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[15]  Huiping Jiang,et al.  Classification of EEG Signal by STFT-CNN Framework: Identification of Right-/left-hand Motor Imagination in BCI Systems , 2017 .

[16]  J. Millán,et al.  Detection of self-paced reaching movement intention from EEG signals , 2012, Front. Neuroeng..

[17]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[18]  Dario Farina,et al.  Detecting and classifying movement-related cortical potentials associated with hand movements in healthy subjects and stroke patients from single-electrode, single-trial EEG , 2015, Journal of neural engineering.

[19]  Mario Ignacio Chacon Murguia,et al.  Classification of multiple motor imagery using deep convolutional neural networks and spatial filters , 2019, Appl. Soft Comput..

[20]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[21]  José del R. Millán,et al.  BNCI Horizon 2020: Towards a Roadmap for the BCI Community , 2015 .

[22]  Mark Hallett,et al.  Exploration of computational methods for classification of movement intention during human voluntary movement from single trial EEG , 2007, Clinical Neurophysiology.

[23]  Yongjae Lee,et al.  Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks , 2020, IEEE Access.

[24]  W. Art Chaovalitwongse,et al.  Pattern Recognition Approaches for Identifying Subcortical Targets during Deep Brain Stimulation Surgery , 2011, IEEE Intelligent Systems.

[25]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  J. Krakauer Motor learning: its relevance to stroke recovery and neurorehabilitation. , 2006, Current opinion in neurology.

[27]  Cristian-Valeriu Soare Brain Computer Interface using Machine Learning , 2016, RoCHI.

[28]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[29]  S. Slobounov,et al.  Rate of force development and the lateralized readiness potential. , 2000, Psychophysiology.

[30]  José Luis Contreras-Vidal,et al.  Reconstructing hand kinematics during reach to grasp movements from electroencephalographic signals , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[31]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[32]  D. Farina,et al.  Detection of movement intention from single-trial movement-related cortical potentials , 2011, Journal of neural engineering.

[33]  J. Fleiss Statistical methods for rates and proportions , 1974 .

[34]  Andrew D. Nordin,et al.  Biomechanics and neural control of movement, 20 years later: what have we learned and what has changed? , 2017, Journal of NeuroEngineering and Rehabilitation.

[35]  Mads Jochumsen,et al.  Detecting and classifying three different hand movement types through electroencephalography recordings for neurorehabilitation , 2015, Medical & Biological Engineering & Computing.

[36]  José Luis Pons Rovira,et al.  Predictive classification of self-paced upper-limb analytical movements with EEG , 2015, Medical & Biological Engineering & Computing.

[37]  Giuseppe Pellizzer,et al.  Brain oscillatory activity during motor preparation: effect of directional uncertainty on beta, but not alpha, frequency band , 2015, Front. Neurosci..

[38]  Stephen Wong,et al.  Optimizing computational feature sets for subthalamic nucleus localization in DBS surgery with feature selection , 2015, Clinical Neurophysiology.

[39]  Emmanuel Maby,et al.  Inferring hand movement kinematics from MEG, EEG and intracranial EEG: From brain-machine interfaces to motor rehabilitation Décoder la cinématique d'un mouvement de la main à partir d'enregistrements MEG et EEG: des interfaces cerveau-machine à la réhabilitation motrice , 2011 .

[40]  Shouqian Sun,et al.  Single-trial EEG classification of motor imagery using deep convolutional neural networks , 2017 .

[41]  Z. Gu,et al.  Decoding hand movement velocity from electroencephalogram signals during a drawing task , 2010, Biomedical engineering online.

[42]  P. Geethanjali,et al.  Time domain Feature extraction and classification of EEG data for Brain Computer Interface , 2012, 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery.

[43]  Mads Jochumsen,et al.  Classification of Hand Grasp Kinetics and Types Using Movement-Related Cortical Potentials and EEG Rhythms , 2017, Comput. Intell. Neurosci..

[44]  M. Hallett,et al.  Classifying EEG signals preceding right hand, left hand, tongue, and right foot movements and motor imageries , 2008, Clinical Neurophysiology.

[45]  O. Arias-Carrión,et al.  EEG-based Brain-Computer Interfaces: An Overview of Basic Concepts and Clinical Applications in Neurorehabilitation , 2010, Reviews in the neurosciences.

[46]  N. Lazar,et al.  Moving to a World Beyond “p < 0.05” , 2019, The American Statistician.

[47]  G.F. Inbar,et al.  Selection of relevant features for classification of movements from single movement-related potentials using a genetic algorithm , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[48]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[49]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[50]  C. Brunia Neural aspects of anticipatory behavior. , 1999, Acta psychologica.

[51]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[52]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[53]  William J. Ray,et al.  Movement-related potentials with reference to isometric force output in discrete and repetitive tasks , 1998, Experimental Brain Research.

[54]  D. Farina,et al.  Precise temporal association between cortical potentials evoked by motor imagination and afference induces cortical plasticity , 2012, The Journal of physiology.

[55]  Seong-Whan Lee,et al.  Decoding Three-Dimensional Trajectory of Executed and Imagined Arm Movements From Electroencephalogram Signals , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[56]  Dario Farina,et al.  Single-trial discrimination of type and speed of wrist movements from EEG recordings , 2009, Clinical Neurophysiology.