Gumpy: a Python toolbox suitable for hybrid brain–computer interfaces

OBJECTIVE The objective of this work is to present gumpy, a new free and open source Python toolbox designed for hybrid brain-computer interface (BCI). APPROACH Gumpy provides state-of-the-art algorithms and includes a rich selection of signal processing methods that have been employed by the BCI community over the last 20 years. In addition, a wide range of classification methods that span from classical machine learning algorithms to deep neural network models are provided. Gumpy can be used for both EEG and EMG biosignal analysis, visualization, real-time streaming and decoding. RESULTS The usage of the toolbox was demonstrated through two different offline example studies, namely movement prediction from EEG motor imagery, and the decoding of natural grasp movements with the applied finger forces from surface EMG (sEMG) signals. Additionally, gumpy was used for real-time control of a robot arm using steady-state visually evoked potentials (SSVEP) as well as for real-time prosthetic hand control using sEMG. Overall, obtained results with the gumpy toolbox are comparable or better than previously reported results on the same datasets. SIGNIFICANCE Gumpy is a free and open source software, which allows end-users to perform online hybrid BCIs and provides different techniques for processing and decoding of EEG and EMG signals. More importantly, the achieved results reveal that gumpy's deep learning toolbox can match or outperform the state-of-the-art in terms of accuracy. This can therefore enable BCI researchers to develop more robust decoding algorithms using novel techniques and hence chart a route ahead for new BCI improvements.

[1]  Gordon Cheng,et al.  Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals , 2018, Sensors.

[2]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[3]  Alin Albu-Schäffer,et al.  The KUKA-DLR Lightweight Robot arm - a new reference platform for robotics research and manufacturing , 2010, ISR/ROBOTIK.

[4]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[5]  Yuanqing Li,et al.  An EEG-based BCI system for 2D cursor control , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[6]  Athanasios V. Vasilakos,et al.  Brain computer interface: control signals review , 2017, Neurocomputing.

[7]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[8]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  M. Nicolelis,et al.  Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. , 2017, Physiological reviews.

[11]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[12]  Klaus-Robert Müller,et al.  Pyff – A Pythonic Framework for Feedback Applications and Stimulus Presentation in Neuroscience , 2010, Front. Neurosci..

[13]  R. Leeb,et al.  BCI Competition 2008 { Graz data set B , 2008 .

[14]  Marco Santello,et al.  Proof of Concept of an Online EMG-Based Decoding of Hand Postures and Individual Digit Forces for Prosthetic Hand Control , 2017, Front. Neurol..

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Yuanqing Li,et al.  A Hybrid BCI System Combining P300 and SSVEP and Its Application to Wheelchair Control , 2013, IEEE Transactions on Biomedical Engineering.

[17]  Ricardo Chavarriaga,et al.  Errare machinale est: the use of error-related potentials in brain-machine interfaces , 2014, Front. Neurosci..

[18]  Benjamin Blankertz,et al.  Mushu, a free- and open source BCI signal acquisition, written in Python , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[19]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[20]  Meng Zhang,et al.  Combined long short-term memory based network employing wavelet coefficients for MI-EEG recognition , 2016, 2016 IEEE International Conference on Mechatronics and Automation.

[21]  Benjamin Blankertz,et al.  Wyrm: A Brain-Computer Interface Toolbox in Python , 2015, Neuroinformatics.

[22]  G. Pfurtscheller,et al.  Brain–Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Yijun Wang,et al.  Brain-Computer Interfaces Based on Visual Evoked Potentials , 2008, IEEE Engineering in Medicine and Biology Magazine.

[24]  Gang Wu,et al.  No impact of transgenic cry1C rice on the rove beetle Paederus fuscipes, a generalist predator of brown planthopper Nilaparvata lugens , 2016, Scientific Reports.

[25]  Bin He,et al.  Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks , 2016, Scientific Reports.

[26]  Mithileysh Sathiyanarayanan,et al.  MYO Armband for physiotherapy healthcare: A case study using gesture recognition application , 2016, 2016 8th International Conference on Communication Systems and Networks (COMSNETS).

[27]  F. Sherwani,et al.  Wavelet based feature extraction for classification of motor imagery signals , 2016, 2016 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES).

[28]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[29]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[30]  Mahyar Hamedi,et al.  Electroencephalographic Motor Imagery Brain Connectivity Analysis for BCI: A Review , 2016, Neural Computation.

[31]  Peter Glöckner,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2013 .

[32]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[33]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[34]  Christian Igel,et al.  An Introduction to Restricted Boltzmann Machines , 2012, CIARP.

[35]  Scott Makeig,et al.  BCILAB: a platform for brain–computer interface development , 2013, Journal of neural engineering.

[36]  Tom Chau,et al.  The roles of predisposing characteristics, established need, and enabling resources on upper extremity prosthesis use and abandonment , 2007, Disability and rehabilitation. Assistive technology.

[37]  Alexandre Barachant,et al.  Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review , 2017 .

[38]  Anatole Lécuyer,et al.  A performance model of selection techniques for p300-based brain-computer interfaces , 2009, CHI.

[39]  Moritz Grosse-Wentrup,et al.  Multiclass Common Spatial Patterns and Information Theoretic Feature Extraction , 2008, IEEE Transactions on Biomedical Engineering.

[40]  Joseph DelPreto,et al.  Plug-and-play supervisory control using muscle and brain signals for real-time gesture and error detection , 2018, Autonomous Robots.

[41]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[42]  Nico M Schmidt,et al.  Online detection of error-related potentials boosts the performance of mental typewriters , 2012, BMC Neuroscience.

[43]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[44]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[45]  M. Buss,et al.  EEG Source Localization for Brain-Computer-Interfaces , 2005, Conference Proceedings. 2nd International IEEE EMBS Conference on Neural Engineering, 2005..

[46]  Guillaume Gibert,et al.  OpenViBE: An Open-Source Software Platform to Design, Test, and Use BrainComputer Interfaces in Real and Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[47]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[48]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[49]  Ricardo Chavarriaga,et al.  A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities , 2011, Journal of neural engineering.

[50]  R. Platts,et al.  Assistive technology in the rehabilitation of patients with high spinal cord lesions , 1993, Paraplegia.

[51]  Joseph DelPreto,et al.  Plug-and-play supervisory control using muscle and brain signals for real-time gesture and error detection , 2020, Auton. Robots.

[52]  Yuanqing Li,et al.  Grouped Automatic Relevance Determination and Its Application in Channel Selection for P300 BCIs , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[53]  Anatole Lécuyer,et al.  Comparative study of band-power extraction techniques for Motor Imagery classification , 2011, 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB).

[54]  Silvestro Micera,et al.  Effective Synchronization of EEG and EMG for Mobile Brain/Body Imaging in Clinical Settings , 2018, Front. Hum. Neurosci..

[55]  Martin Luessi,et al.  MEG and EEG data analysis with MNE-Python , 2013, Front. Neuroinform..

[56]  Elvira Pirondini,et al.  EMG-based decoding of grasp gestures in reaching-to-grasping motions , 2017, Robotics Auton. Syst..