Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review

Despite its short history, the use of Riemannian geometry in brain-computer interface (BCI) decoding is currently attracting increasing attention, due to accumulating documentation of its simplicity, accuracy, robustness and transfer learning capabilities, including the winning score obtained in five recent international predictive modeling BCI data competitions. The Riemannian framework is sharp from a mathematical perspective, yet in practice it is simple, both algorithmically and computationally. This allows the conception of online decoding machines suiting real-world operation in adverse conditions. We provide here a review on the use of Riemannian geometry for BCI and a primer on the classification frameworks based on it. While the theoretical research on Riemannian geometry is technical, our aim here is to show the appeal of the framework on an intuitive geometrical ground. In particular, we provide a rationale for its robustness and transfer learning capabilities and we elucidate the link between a simple Riemannian classifier and a state-of-the-art spatial filtering approach. We conclude by reporting details on the construction of data points to be manipulated in the Riemannian framework in the context of BCI and by providing links to available open-source Matlab and Python code libraries for designing BCI decoders.

[1]  Marco Congedo,et al.  EEG Source Analysis , 2013 .

[2]  Saeed Shiry Ghidary,et al.  Dimensionality reduction based on Distance Preservation to Local Mean (DPLM) for SPD matrices and its application in BCI , 2016, ArXiv.

[3]  F. Cincotti,et al.  Eye-gaze independent EEG-based brain–computer interfaces for communication , 2012, Journal of neural engineering.

[4]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[5]  M. Congedo,et al.  The Riemannian Potato: an automatic and adaptive artifact detection method for online experiments using Riemannian geometry , 2013 .

[6]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[7]  Ricardo Chavarriaga,et al.  Quantifying Electrode Reliability During Brain–Computer Interface Operation , 2015, IEEE Transactions on Biomedical Engineering.

[8]  Andreas Karlsson,et al.  Matrix Analysis for Statistics , 2007, Technometrics.

[9]  Anton Nijholt,et al.  Usability of Three Electroencephalogram Headsets for Brain-Computer Interfaces: A Within Subject Comparison , 2015, Interact. Comput..

[10]  R. Fisher 036: On a Distribution Yielding the Error Functions of Several Well Known Statistics. , 1924 .

[11]  Qi Wu,et al.  Epileptic Seizure Detection with Log-Euclidean Gaussian Kernel-Based Sparse Representation , 2016, Int. J. Neural Syst..

[12]  Xiaoming Huo A statistical analysis of Fukunaga-Koontz transform , 2004, IEEE Signal Processing Letters.

[13]  P. London Injury , 1969, Definitions.

[14]  T. Parsons,et al.  Brain–computer interface targeting non-motor functions after spinal cord injury: a case report , 2015, Spinal Cord.

[15]  Jijun Tong,et al.  Multi-phase cycle coding for SSVEP based brain-computer interfaces , 2015, Biomedical engineering online.

[16]  Xiaofeng Xie,et al.  Motor Imagery Classification Based on Bilinear Sub-Manifold Learning of Symmetric Positive-Definite Matrices , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  C. Jutten,et al.  Classification de potentiels évoqués P300 par géométrie riemannienne pour les interfaces cerveau-machine EEG , 2013 .

[18]  Fabrizio de Vico Fallani,et al.  Riemannian Geometry Applied to Detection of Respiratory States From EEG Signals: The Basis for a Brain–Ventilator Interface , 2016, IEEE Transactions on Biomedical Engineering.

[19]  Ricardo Chavarriaga,et al.  Workshops of the Fifth International Brain-Computer Interface Meeting: Defining the Future. , 2014, Brain computer interfaces.

[20]  Xiaorong Gao,et al.  Frequency and Phase Mixed Coding in SSVEP-Based Brain--Computer Interface , 2011, IEEE Transactions on Biomedical Engineering.

[21]  C. Jutten,et al.  A Brain-Switch using Riemannian Geometry , 2011 .

[22]  Tobias Kaufmann,et al.  Spelling is Just a Click Away – A User-Centered Brain–Computer Interface Including Auto-Calibration and Predictive Text Entry , 2012, Front. Neurosci..

[23]  Christian Jutten,et al.  " Brain Invaders": a prototype of an open-source P300-based video game working with the OpenViBE platform , 2011 .

[24]  Alexandre Barachant,et al.  A special form of SPD covariance matrix for interpretation and visualization of data manipulated with Riemannian geometry , 2014 .

[25]  Toshihisa Tanaka,et al.  Robust Averaging of Covariance Matrices by Riemannian Geometry for Motor-Imagery Brain–Computer Interfacing , 2016 .

[26]  Andrea Kübler,et al.  Empathy, motivation, and P300 BCI performance , 2013, Front. Hum. Neurosci..

[27]  Maher Moakher,et al.  Symmetric Positive-Definite Matrices: From Geometry to Applications and Visualization , 2006, Visualization and Processing of Tensor Fields.

[28]  Kon Max Wong,et al.  Signal classification by power spectral density: An approach via Riemannian geometry , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[29]  Gert Pfurtscheller,et al.  Overt foot movement detection in one single Laplacian EEG derivation , 2008, Journal of Neuroscience Methods.

[30]  Masashi Sugiyama,et al.  Geometry-aware stationary subspace analysis , 2016, ACML.

[31]  T Verhoeven,et al.  Towards a symbiotic brain–computer interface: exploring the application–decoder interaction , 2015, Journal of neural engineering.

[32]  J. Schmee An Introduction to Multivariate Statistical Analysis , 1986 .

[33]  Masashi Sugiyama,et al.  Geometry-aware principal component analysis for symmetric positive definite matrices , 2017, Machine Learning.

[34]  Jun Lu,et al.  Adaptive Spatio-Temporal Filtering for Movement Related Potentials in EEG-Based Brain–Computer Interfaces , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Andrzej Cichocki,et al.  Whether generic model works for rapid ERP-based BCI calibration , 2013, Journal of Neuroscience Methods.

[36]  Benjamin Blankertz,et al.  Large-Scale Assessment of a Fully Automatic Co-Adaptive Motor Imagery-Based Brain Computer Interface , 2016, PloS one.

[37]  Niels Birbaumer,et al.  Communication in locked-in state after brainstem stroke: a brain-computer-interface approach. , 2015, Annals of translational medicine.

[38]  Mehrtash Tafazzoli Harandi,et al.  From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matrices , 2014, ECCV.

[39]  M. Congedo,et al.  Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices , 2015, PloS one.

[40]  Peng Yuan,et al.  A study of the existing problems of estimating the information transfer rate in online brain–computer interfaces , 2013, Journal of neural engineering.

[41]  Daniel P. Ferris,et al.  Weighted phase lag index stability as an artifact resistant measure to detect cognitive EEG activity during locomotion , 2012, Journal of NeuroEngineering and Rehabilitation.

[42]  Stephanie Brandl,et al.  Robust common spatial patterns based on Bhattacharyya distance and Gamma divergence , 2015, The 3rd International Winter Conference on Brain-Computer Interface.

[43]  Benjamin Schrauwen,et al.  A Bayesian Model for Exploiting Application Constraints to Enable Unsupervised Training of a P300-based BCI , 2012, PloS one.

[44]  José del R. Millán,et al.  Brain-Computer Interfaces , 2020, Handbook of Clinical Neurology.

[45]  N. Birbaumer,et al.  The Influence of Psychological State and Motivation on Brain–Computer Interface Performance in Patients with Amyotrophic Lateral Sclerosis – a Longitudinal Study , 2010, Front. Neuropharma..

[46]  Klaus-Robert Müller,et al.  The LDA beamformer: Optimal estimation of ERP source time series using linear discriminant analysis , 2016, NeuroImage.

[47]  Marco Congedo,et al.  Robust Brain-computer Interface for virtual Keyboard (RoBIK): Project results , 2013 .

[48]  Stefan Haufe,et al.  Optimizing event-related potential based brain-computer interfaces: a systematic evaluation of dynamic stopping methods. , 2013, Journal of neural engineering.

[49]  D. L. Schomer,et al.  Niedermeyer's Electroencephalography: Basic Principles, Clinical Applications, and Related Fields , 2012 .

[50]  K. M. Wong,et al.  EEG signal classification based on a Riemannian distance measure , 2009, 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH).

[51]  M Congedo,et al.  sw-SVM: sensor weighting support vector machines for EEG-based brain–computer interfaces , 2011, Journal of neural engineering.

[52]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[53]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[54]  Christian Jutten,et al.  On the blind source separation of human electroencephalogram by approximate joint diagonalization of second order statistics , 2008, Clinical Neurophysiology.

[55]  Ronald Phlypo,et al.  Fixed Point Algorithms for Estimating Power Means of Positive Definite Matrices , 2016, IEEE Transactions on Signal Processing.

[56]  Noboru Nakamura,et al.  Geometric Means of Positive Operators , 2009 .

[57]  Egor Krivov,et al.  Dimensionality reduction with isomap algorithm for EEG covariance matrices , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[58]  J. Wolpaw,et al.  Brain-computer communication: unlocking the locked in. , 2001, Psychological bulletin.

[59]  José del R. Millán,et al.  BNCI Horizon 2020: Towards a Roadmap for the BCI Community , 2015 .

[60]  Keinosuke Fukunaga,et al.  Application of the Karhunen-Loève Expansion to Feature Selection and Ordering , 1970, IEEE Trans. Computers.

[61]  A. Barachant,et al.  Extraction of motor patterns from joint EEG/EMG recording: A Riemannian Geometry approach , 2016 .

[62]  Rajendra Bhatia,et al.  The Riemannian Mean of Positive Matrices , 2013 .

[63]  Ying Sun,et al.  Adaptation in P300 Brain–Computer Interfaces: A Two-Classifier Cotraining Approach , 2010, IEEE Transactions on Biomedical Engineering.

[64]  Christian Jutten,et al.  Multiclass Brain–Computer Interface Classification by Riemannian Geometry , 2012, IEEE Transactions on Biomedical Engineering.

[65]  R. Bhatia Positive Definite Matrices , 2007 .

[66]  Cuntai Guan,et al.  High performance P300 speller for brain-computer interface , 2004, IEEE International Workshop on Biomedical Circuits and Systems, 2004..

[67]  Andreas M. Ray,et al.  A subject-independent pattern-based Brain-Computer Interface , 2015, Front. Behav. Neurosci..

[68]  Marco Congedo,et al.  Spatio-temporal common pattern: A companion method for ERP analysis in the time domain , 2016, Journal of Neuroscience Methods.

[69]  Jonathan H. Manton,et al.  A globally convergent numerical algorithm for computing the centre of mass on compact Lie groups , 2004, ICARCV 2004 8th Control, Automation, Robotics and Vision Conference, 2004..

[70]  Toshihisa Tanaka,et al.  Dimensionality reduction of sample covariance matrices by graph fourier transform for motor imagery brain-machine interface , 2016, 2016 IEEE Statistical Signal Processing Workshop (SSP).

[71]  Suvrit Sra,et al.  A new metric on the manifold of kernel matrices with application to matrix geometric means , 2012, NIPS.

[72]  Kon Max Wong,et al.  Electroencephalogram signals classification for sleep-state decision - a Riemannian geometry approach , 2012, IET Signal Process..

[73]  Gert Pfurtscheller,et al.  Brain-computer interface: a new communication device for handicapped persons , 1993 .

[74]  B.Z. Allison,et al.  ERPs evoked by different matrix sizes: implications for a brain computer interface (BCI) system , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[75]  Gunnar Rätsch,et al.  Prototype Classification: Insights from Machine Learning , 2009, Neural Computation.

[76]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[77]  Jonathan R. Wolpaw,et al.  Brain–Computer InterfacesPrinciples and Practice , 2012 .

[78]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[79]  Takashi Hanakawa,et al.  Neuroanatomical correlates of brain–computer interface performance , 2015, NeuroImage.

[80]  J. W. Minett,et al.  Optimizing the P300-based brain–computer interface: current status, limitations and future directions , 2011, Journal of neural engineering.

[81]  Chang S. Nam,et al.  Effects of Luminosity Contrast and Stimulus Duration on User Performance and Preference in a P300-Based Brain–Computer Interface , 2014, Int. J. Hum. Comput. Interact..

[82]  Christian Jutten,et al.  Classification of covariance matrices using a Riemannian-based kernel for BCI applications , 2013, Neurocomputing.

[83]  Vicenç Gómez,et al.  Adaptive Multiclass Classification for Brain Computer Interfaces , 2014, Neural Computation.

[84]  Bernhard Schölkopf,et al.  Transfer Learning in Brain-Computer Interfaces , 2015, IEEE Computational Intelligence Magazine.

[85]  Motoaki Kawanabe,et al.  Robust common spatial patterns by minimum divergence covariance estimator , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[86]  Z J Koles,et al.  EEG source localization: implementing the spatio-temporal decomposition approach. , 1998, Electroencephalography and clinical neurophysiology.

[87]  Yijun Wang,et al.  A high-speed BCI based on code modulation VEP , 2011, Journal of neural engineering.

[88]  Ivan Volosyak,et al.  Evaluation of the Bremen SSVEP based BCI in real world conditions , 2009, 2009 IEEE International Conference on Rehabilitation Robotics.

[89]  Jason Farquhar,et al.  Classifying Regularized Sensor Covariance Matrices: An Alternative to CSP , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[90]  Mehrtash Harandi,et al.  Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[91]  Maureen Clerc,et al.  Comparison of Hierarchical and Non-Hierarchical Classification for Motor Imagery Based BCI Systems , 2016 .

[92]  Christian Jutten,et al.  Single-trial classification of multi-user P300-based Brain-Computer Interface using riemannian geometry , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[93]  Jyh-Yeong Chang,et al.  Novel Dry Polymer Foam Electrodes for Long-Term EEG Measurement , 2011, IEEE Transactions on Biomedical Engineering.

[94]  Klaus-Robert Müller,et al.  True Zero-Training Brain-Computer Interfacing – An Online Study , 2014, PloS one.

[95]  J. Wolpaw,et al.  P300-based brain-computer interface (BCI) event-related potentials (ERPs): People with amyotrophic lateral sclerosis (ALS) vs. age-matched controls , 2015, Clinical Neurophysiology.

[96]  Motoaki Kawanabe,et al.  Divergence-Based Framework for Common Spatial Patterns Algorithms , 2014, IEEE Reviews in Biomedical Engineering.

[97]  Yueqing Li Evaluation of Collaborative Brain-Computer Interface (C-BCI) for People with Motor Disabilities. , 2014 .

[98]  Masashi Sugiyama,et al.  Supervised LogEuclidean Metric Learning for Symmetric Positive Definite Matrices , 2015, ArXiv.

[99]  Pierre Brémaud Fourier Analysis of Time Series , 2014 .

[100]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[101]  Z J Koles,et al.  The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. , 1991, Electroencephalography and clinical neurophysiology.

[102]  Addison W. Bohannon,et al.  Spectral Transfer Learning Using Information Geometry for a User-Independent Brain-Computer Interface , 2016, Front. Neurosci..

[103]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[104]  Selina Wriessnegger,et al.  Automatic Pause Detection during P300 Web Browsing , 2014 .

[105]  A. Kübler,et al.  Brain Painting: First Evaluation of a New Brain–Computer Interface Application with ALS-Patients and Healthy Volunteers , 2010, Front. Neurosci..

[106]  Benjamin Schrauwen,et al.  Dynamic stopping in a calibration-less P300 speller , 2013 .

[107]  Christian Jutten,et al.  Riemannian Geometry Applied to BCI Classification , 2010, LVA/ICA.

[108]  Brian R. Tietz,et al.  Deciding Which Way to Go: How Do Insects Alter Movements to Negotiate Barriers? , 2012, Front. Neurosci..

[109]  Ivo Käthner,et al.  Comparison of eye tracking, electrooculography and an auditory brain-computer interface for binary communication: a case study with a participant in the locked-in state , 2015, Journal of NeuroEngineering and Rehabilitation.

[110]  Saeed Shiry Ghidary,et al.  Kernel learning over the manifold of symmetric positive definite matrices for dimensionality reduction in a BCI application , 2016, Neurocomputing.

[111]  Alexandre Barachant,et al.  Brain-computer interface for the communication of acute patients: a feasibility study and a randomized controlled trial comparing performance with healthy participants and a traditional assistive device , 2016 .

[112]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[113]  Tomasz M. Rutkowski Data–Driven Multimodal Sleep Apnea Events Detection , 2016, Journal of Medical Systems.

[114]  Hasan Onur Keles,et al.  Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks , 2016, PloS one.

[115]  Xingyu Wang,et al.  Optimized stimulus presentation patterns for an event-related potential EEG-based brain–computer interface , 2011, Medical & Biological Engineering & Computing.

[116]  Global EEG synchronization as an indicator of emotional arousal and its application for tracking emotional changes during video watching , 2014 .

[117]  J. Wolpaw,et al.  A novel P300-based brain–computer interface stimulus presentation paradigm: Moving beyond rows and columns , 2010, Clinical Neurophysiology.

[118]  Stéphane Bonnet,et al.  Detection of mental fatigue using an active BCI inspired signal processing chain , 2014 .

[119]  Benjamin Blankertz,et al.  Presenting a Spatial-Geometric EEG Feature to Classify BMD and Schizophrenic Patients , 2016 .

[120]  Anatole Lécuyer,et al.  Author manuscript, published in "IEEE Transactions on Computational Intelligence and AI in games (2013)" Two Brains, One Game: Design and Evaluation of a Multi-User BCI Video Game Based on Motor Imagery , 2022 .

[121]  Steven L. Johnson,et al.  A P300-Based Brain–Computer Interface: Effects of Interface Type and Screen Size , 2010, Int. J. Hum. Comput. Interact..

[122]  Christian Jutten,et al.  "Brain Invaders 2" : an open source Plug & Play multi-user BCI videogame , 2016 .

[123]  F. Lotte,et al.  Self-Paced Brain-Computer Interaction with Virtual Worlds: A Quantitative and Qualitative Study "Out of the Lab" , 2008 .

[124]  Christian Jutten,et al.  Common Spatial Pattern revisited by Riemannian geometry , 2010, 2010 IEEE International Workshop on Multimedia Signal Processing.

[125]  Masashi Sugiyama,et al.  Averaging covariance matrices for EEG signal classification based on the CSP: An empirical study , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[126]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..

[127]  D. Orlikowski,et al.  A comparison of recording modalities of P300 event-related potentials (ERP) for brain-computer interface (BCI) paradigm , 2013, Neurophysiologie Clinique/Clinical Neurophysiology.

[128]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[129]  Emmanuel K. Kalunga,et al.  Data augmentation in Riemannian space for Brain-Computer Interfaces , 2015 .

[130]  T. W. Anderson An Introduction to Multivariate Statistical Analysis , 1959 .

[131]  Niels Birbaumer,et al.  Residual Upper Arm Motor Function Primes Innervation of Paretic Forearm Muscles in Chronic Stroke after Brain-Machine Interface (BMI) Training , 2015, PloS one.

[132]  Vicenç Gómez,et al.  Adaptive Classification on Brain-Computer Interfaces Using Reinforcement Signals , 2012, Neural Computation.

[133]  Florian Yger,et al.  A review of kernels on covariance matrices for BCI applications , 2013, 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[134]  Chang S. Nam,et al.  The Human Factors and Ergonomics of P300-Based Brain-Computer Interfaces , 2015, Brain sciences.