Continuous low-frequency EEG decoding of arm movement for closed-loop, natural control of a robotic arm

OBJECTIVE Continuous decoding of voluntary movement is desirable for closed-loop, natural control of neuroprostheses. Recent studies showed the possibility to reconstruct the hand trajectories from low-frequency (LF) electroencephalographic (EEG) signals. So far this has only been performed offline. Here, we attempt for the first time continuous online control of a robotic arm with LF-EEG-based decoded movements. APPROACH The study involved ten healthy participants, asked to track a moving target by controlling a robotic arm. At the beginning of the experiment, the robot was fully controlled by the participant's hand trajectories. After calibrating the decoding model, that control was gradually replaced by LF-EEG-based decoded trajectories, first with 33%, 66% and finally 100% EEG control. Likewise with other offline studies, we regressed the movement parameters (two-dimensional positions, velocities, and accelerations) from the EEG with partial least squares (PLS) regression. To integrate the information from the different movement parameters, we introduced a combined PLS and Kalman filtering approach (named PLSKF). MAIN RESULTS We obtained moderate yet overall significant (α = 0.05) online correlations between hand kinematics and PLSKF-decoded trajectories of 0.32 on average. With respect to PLS regression alone, the PLSKF had a stable correlation increase of Δr = 0.049 on average, demonstrating the successful integration of different models. Parieto-occipital activations were highlighted for the velocity and acceleration decoder patterns. The level of robot control was above chance in all conditions. Participants finally reported to feel enough control to be able to improve with training, even in the 100% EEG condition. SIGNIFICANCE Continuous low frequency EEG-based movement decoding for the online control of a robotic arm was achieved for the first time. The potential bottlenecks arising when switching from offline to online decoding, and possible solutions, were described. The effect of the PLSKF and its extensibility to different experimental designs were discussed.

[1]  Trent J. Bradberry,et al.  Reconstructing Three-Dimensional Hand Movements from Noninvasive Electroencephalographic Signals , 2010, The Journal of Neuroscience.

[2]  José Luis Contreras-Vidal,et al.  Decoding center-out hand velocity from MEG signals during visuomotor adaptation , 2009, NeuroImage.

[3]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[4]  Z. Gu,et al.  Decoding hand movement velocity from electroencephalogram signals during a drawing task , 2010, Biomedical engineering online.

[5]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[6]  A. Aertsen,et al.  Predominance of Movement Speed Over Direction in Neuronal Population Signals of Motor Cortex: Intracranial EEG Data and A Simple Explanatory Model , 2016, Cerebral cortex.

[7]  Andreas Schulze-Bonhage,et al.  Prediction of arm movement trajectories from ECoG-recordings in humans , 2008, Journal of Neuroscience Methods.

[8]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[9]  Richard A Andersen,et al.  Decoding Trajectories from Posterior Parietal Cortex Ensembles , 2008, The Journal of Neuroscience.

[10]  Jonathan D. Nelson,et al.  Multiple Parietal Reach Regions in Humans: Cortical Representations for Visual and Proprioceptive Feedback during On-Line Reaching , 2009, The Journal of Neuroscience.

[11]  Gernot R Müller-Putz,et al.  Tuning characteristics of low-frequency EEG to positions and velocities in visuomotor and oculomotor tracking tasks , 2018, Scientific Reports.

[12]  Y. Benjamini,et al.  Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics , 1999 .

[13]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[14]  A. P. Georgopoulos,et al.  Cortical mechanisms related to the direction of two-dimensional arm movements: relations in parietal area 5 and comparison with motor cortex , 1983, Experimental Brain Research.

[15]  C. Braun,et al.  A review on directional information in neural signals for brain-machine interfaces , 2009, Journal of Physiology-Paris.

[16]  A. P. Vinod,et al.  Noninvasive Brain-Computer Interface: Decoding Arm Movement Kinematics and Motor Control , 2016, IEEE Systems, Man, and Cybernetics Magazine.

[17]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[18]  N. Birbaumer,et al.  On the Usage of Linear Regression Models to Reconstruct Limb Kinematics from Low Frequency EEG Signals , 2013, PloS one.

[19]  Miriam Zacksenhouse,et al.  Cortical Ensemble Adaptation to Represent Velocity of an Artificial Actuator Controlled by a Brain-Machine Interface , 2005, The Journal of Neuroscience.

[20]  Paul B. Johnson,et al.  Making arm movements within different parts of space: dynamic aspects in the primate motor cortex , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  Kenneth F. Valyear,et al.  Human parietal cortex in action , 2006, Current Opinion in Neurobiology.

[22]  J. Wolpaw,et al.  Decoding two-dimensional movement trajectories using electrocorticographic signals in humans , 2007, Journal of neural engineering.

[23]  Andreea Ioana Sburlea,et al.  HEAR to remove pops and drifts: the high-variance electrode artifact removal (HEAR) algorithm , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[24]  Andreea Ioana Sburlea,et al.  Using machine learning to reveal the population vector from EEG signals , 2020, Journal of neural engineering.

[25]  A. Georgopoulos,et al.  Magnetoencephalographic signals predict movement trajectory in space , 2005, Experimental Brain Research.

[26]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[27]  Olivier D. Faugeras,et al.  A common formalism for the Integral formulations of the forward EEG problem , 2005, IEEE Transactions on Medical Imaging.

[28]  C. Braun,et al.  Hand Movement Direction Decoded from MEG and EEG , 2008, The Journal of Neuroscience.

[29]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[30]  Gernot R. Müller-Putz,et al.  Using a Noninvasive Decoding Method to Classify Rhythmic Movement Imaginations of the Arm in Two Planes , 2015, IEEE Transactions on Biomedical Engineering.

[31]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[32]  L. Paninski,et al.  Spatiotemporal tuning of motor cortical neurons for hand position and velocity. , 2004, Journal of neurophysiology.

[33]  S. D. Jong SIMPLS: an alternative approach to partial least squares regression , 1993 .

[34]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[35]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[36]  Gernot R. Müller-Putz,et al.  Decoding of velocities and positions of 3D arm movement from EEG , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[37]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[38]  Ricardo Chavarriaga,et al.  Classification of upper limb center-out reaching tasks by means of EEG-based continuous decoding techniques , 2017, Journal of NeuroEngineering and Rehabilitation.

[39]  Jonathan D. Nelson,et al.  Human cortical representations for reaching: Mirror neurons for execution, observation, and imagery , 2007, NeuroImage.

[40]  Théodore Papadopoulo,et al.  OpenMEEG: opensource software for quasistatic bioelectromagnetics , 2010, Biomedical engineering online.

[41]  S. Wold,et al.  PLS-regression: a basic tool of chemometrics , 2001 .

[42]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[43]  Francis R. Willett,et al.  Restoration of reaching and grasping in a person with tetraplegia through brain-controlled muscle stimulation: a proof-of-concept demonstration , 2017, The Lancet.

[44]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[45]  D. Louis Collins,et al.  Unbiased average age-appropriate atlases for pediatric studies , 2011, NeuroImage.

[46]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[47]  Fabien Lotte,et al.  Towards improved BCI based on human learning principles , 2015, The 3rd International Winter Conference on Brain-Computer Interface.

[48]  A. Benabid,et al.  An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: a proof-of-concept demonstration , 2019, The Lancet Neurology.

[49]  Dimitrios Pantazis,et al.  Coherent neural representation of hand speed in humans revealed by MEG imaging , 2007, Proceedings of the National Academy of Sciences.

[50]  M. Nicolelis,et al.  Unscented Kalman Filter for Brain-Machine Interfaces , 2009, PloS one.

[51]  Gernot R Müller-Putz,et al.  Upper limb movements can be decoded from the time-domain of low-frequency EEG , 2017, PloS one.

[52]  Christoph M. Michel,et al.  Towards the utilization of EEG as a brain imaging tool , 2012, NeuroImage.

[53]  Andreea Ioana Sburlea,et al.  Corneo-retinal-dipole and eyelid-related eye artifacts can be corrected offline and online in electroencephalographic and magnetoencephalographic signals , 2020, NeuroImage.

[54]  G R Müller-Putz,et al.  From classic motor imagery to complex movement intention decoding: The noninvasive Graz-BCI approach. , 2016, Progress in brain research.

[55]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[56]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[57]  Andrés Úbeda,et al.  Assessing Movement Factors in Upper Limb Kinematics Decoding from EEG Signals , 2015, PloS one.

[58]  M. Murray,et al.  EEG source imaging , 2004, Clinical Neurophysiology.