Low-latency multi-threaded processing of neuronal signals for brain-computer interfaces

Brain-computer interfaces (BCIs) require demanding numerical computations to transfer brain signals into control signals driving an external actuator. Increasing the computational performance of the BCI algorithms carrying out these calculations enables faster reaction to user inputs and allows using more demanding decoding algorithms. Here we introduce a modular and extensible software architecture with a multi-threaded signal processing pipeline suitable for BCI applications. The computational load and latency (the time that the system needs to react to user input) are measured for different pipeline implementations in typical BCI applications with realistic parameter settings. We show that BCIs can benefit substantially from the proposed parallelization: firstly, by reducing the latency and secondly, by increasing the amount of recording channels and signal features that can be used for decoding beyond the amount which can be handled by a single thread. The proposed software architecture provides a simple, yet flexible solution for BCI applications.

[1]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[2]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[3]  S. Haykin Kalman Filtering and Neural Networks , 2001 .

[4]  Andrew William Roscoe,et al.  The Theory and Practice of Concurrency , 1997 .

[5]  D. Marr,et al.  Hyper-Threading Technology Architecture and MIcroarchitecture , 2002 .

[6]  J. Blumberg,et al.  Adaptive Classification for Brain Computer Interfaces , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Y. Istefanopulos,et al.  IEEE Engineering in Medicine and Biology Society , 2019, IEEE Transactions on Biomedical Engineering.

[8]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[9]  Vikash Gilja,et al.  A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces. , 2011, Journal of neurophysiology.

[10]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[11]  Michael J. Black,et al.  Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[13]  M. E. Galassi,et al.  GNU SCIENTI C LIBRARY REFERENCE MANUAL , 2005 .

[14]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[15]  Wei Wu,et al.  Real-Time Decoding of Nonstationary Neural Activity in Motor Cortex , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  C. Mehring,et al.  Encoding of Movement Direction in Different Frequency Ranges of Motor Cortical Local Field Potentials , 2005, The Journal of Neuroscience.

[17]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[18]  Qing Li,et al.  Unified Modeling Language , 2009 .

[19]  Jerald D. Kralik,et al.  Chronic, multisite, multielectrode recordings in macaque monkeys , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Justin C. Williams,et al.  Massively Parallel Signal Processing using the Graphics Processing Unit for Real-Time Brain–Computer Interface Feature Extraction , 2009, Front. Neuroeng..

[21]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[22]  Klaus Pawelzik,et al.  Towards On-line Adaptation of Neuro-prostheses with Neuronal Evaluation Signals , 2006, Biological Cybernetics.

[23]  C. Mehring,et al.  Detection of Error Related Neuronal Responses Recorded by Electrocorticography in Humans during Continuous Movements , 2013, PloS one.

[24]  Tamar Frankel [The theory and the practice...]. , 2001, Tijdschrift voor diergeneeskunde.

[25]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[26]  Douglas A. Wolfe,et al.  Nonparametric Statistical Methods , 1973 .

[27]  Shun-ichi Amari,et al.  A Theory of Pattern Recognition , 1968 .

[28]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[29]  Bjarne Stroustrup,et al.  C++ Programming Language , 1986, IEEE Softw..

[30]  José Carlos Príncipe,et al.  Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans: Motor control features , 2009, Neural Networks.

[31]  Dean J Krusienski,et al.  Brain-computer interfaces in medicine. , 2012, Mayo Clinic proceedings.

[32]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[33]  Jont B. Allen,et al.  Short term spectral analysis, synthesis, and modification by discrete Fourier transform , 1977 .

[34]  Eunice E. Santos,et al.  Efficient and Optimal Parallel Algorithms for Cholesky Decomposition , 2003, J. Math. Model. Algorithms.

[35]  Martin Gogolla Unified Modeling Language , 2009, Encyclopedia of Database Systems.

[36]  Eilon Vaadia,et al.  Kernel-ARMA for Hand Tracking and Brain-Machine interfacing During 3D Motor Control , 2008, NIPS.

[37]  Subhabrata Chakraborti,et al.  Nonparametric Statistical Inference , 2011, International Encyclopedia of Statistical Science.

[38]  Jaime Gómez Gil,et al.  Brain Computer Interfaces, a Review , 2012, Sensors.

[39]  Michael J. Black,et al.  A quantitative comparison of linear and non-linear models of motor cortical activity for the encoding and decoding of arm motions , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[40]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[41]  Arjun K. Bansal,et al.  Decoding 3D reach and grasp from hybrid signals in motor and premotor cortices: spikes, multiunit activity, and local field potentials. , 2012, Journal of neurophysiology.

[42]  Ivar Jacobson,et al.  Unified Modeling Language User Guide, The (2nd Edition) (Addison-Wesley Object Technology Series) , 2005 .

[43]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[44]  Jörg Fischer,et al.  An online brain–machine interface using decoding of movement direction from the human electrocorticogram , 2012, Journal of neural engineering.

[45]  Andrzej Cichocki,et al.  Novel Features for Brain-Computer Interfaces , 2007, Comput. Intell. Neurosci..

[46]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[47]  Martin Fowler,et al.  Patterns of Enterprise Application Architecture , 2002 .

[48]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[49]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .