How to Build a Hybrid Neurofeedback Platform Combining EEG and fMRI

Multimodal neurofeedback estimates brain activity using information acquired with more than one neurosignal measurement technology. In this paper we describe how to set up and use a hybrid platform based on simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), then we illustrate how to use it for conducting bimodal neurofeedback experiments. The paper is intended for those willing to build a multimodal neurofeedback system, to guide them through the different steps of the design, setup, and experimental applications, and help them choose a suitable hardware and software configuration. Furthermore, it reports practical information from bimodal neurofeedback experiments conducted in our lab. The platform presented here has a modular parallel processing architecture that promotes real-time signal processing performance and simple future addition and/or replacement of processing modules. Various unimodal and bimodal neurofeedback experiments conducted in our lab showed high performance and accuracy. Currently, the platform is able to provide neurofeedback based on electroencephalography and functional magnetic resonance imaging, but the architecture and the working principles described here are valid for any other combination of two or more real-time brain activity measurement technologies.

[1]  S. Bunce,et al.  Functional near-infrared spectroscopy , 2006, IEEE Engineering in Medicine and Biology Magazine.

[2]  Sven Haller,et al.  Real-time fMRI feedback training may improve chronic tinnitus , 2010, European Radiology.

[3]  K. Shadan,et al.  Available online: , 2012 .

[4]  Bettina Sorger,et al.  Windowed Correlation: A Suitable Tool for Providing Dynamic fMRI-Based Functional Connectivity Neurofeedback on Task Difficulty , 2014, PloS one.

[5]  Andrzej Cichocki,et al.  Removal of ballistocardiogram artifacts from simultaneously recorded EEG and fMRI data using independent component analysis , 2006, IEEE Transactions on Biomedical Engineering.

[6]  Arno Villringer,et al.  Internal ventilation system of MR scanners induces specific EEG artifact during simultaneous EEG-fMRI , 2013, NeuroImage.

[7]  Christian Barillot,et al.  An a contrario approach for the detection of patient-specific brain perfusion abnormalities with arterial spin labelling , 2016, NeuroImage.

[8]  Klaus Mathiak,et al.  Signal quality and Bayesian signal processing in neurofeedback based on real-time fMRI , 2012, NeuroImage.

[9]  Abdulhamit Subasi,et al.  EEG signal classification using PCA, ICA, LDA and support vector machines , 2010, Expert Syst. Appl..

[10]  Vince D. Calhoun,et al.  Neuronal chronometry of target detection: Fusion of hemodynamic and event-related potential data , 2005, NeuroImage.

[11]  Lorenzo Bruzzone,et al.  The Use of a priori Information in ICA-Based Techniques for Real-Time fMRI: An Evaluation of Static/Dynamic and Spatial/Temporal Characteristics , 2013, Front. Hum. Neurosci..

[12]  Xu Cui,et al.  Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics , 2010, NeuroImage.

[13]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[14]  Nathan Intrator,et al.  An EEG Finger-Print of fMRI deep regional activation , 2014, NeuroImage.

[15]  Kayako Matsuo,et al.  Dynamic monitoring of brain activation under visual stimulation using fMRI—The advantage of real-time fMRI with sliding window GLM analysis , 2006, Journal of Neuroscience Methods.

[16]  Nathan Intrator,et al.  Limbic Activity Modulation Guided by Functional Magnetic Resonance Imaging–Inspired Electroencephalography Improves Implicit Emotion Regulation , 2016, Biological Psychiatry.

[17]  Paul L. Nunez,et al.  The surface laplacian, high resolution EEG and controversies , 2005, Brain Topography.

[18]  Bart Vanrumste,et al.  Journal of Neuroengineering and Rehabilitation Open Access Review on Solving the Inverse Problem in Eeg Source Analysis , 2022 .

[19]  Lorenzo Bruzzone,et al.  ICA analysis of fMRI with real-time constraints: an evaluation of fast detection performance as function of algorithms, parameters and a priori conditions , 2013, Front. Hum. Neurosci..

[20]  P. Federico.  Simultaneous Eeg and Fmri: Recording, Analysis and Application , 2010, Neurology.

[21]  S. W. Roberts,et al.  Control Chart Tests Based on Geometric Moving Averages , 2000, Technometrics.

[22]  Rainer Goebel,et al.  Real-time independent component analysis of fMRI time-series , 2003, NeuroImage.

[23]  Louis Lemieux,et al.  Identification of EEG Events in the MR Scanner: The Problem of Pulse Artifact and a Method for Its Subtraction , 1998, NeuroImage.

[24]  Vince D. Calhoun,et al.  Mind over chatter: Plastic up-regulation of the fMRI salience network directly after EEG neurofeedback , 2013, NeuroImage.

[25]  Kevin A. Johnson,et al.  Volitional reduction of anterior cingulate cortex activity produces decreased cue craving in smoking cessation: a preliminary real‐time fMRI study , 2013, Addiction biology.

[26]  Heidi Johansen-Berg,et al.  Faculty of 1000 evaluation for Acquired control of ventral premotor cortex activity by feedback training: an exploratory real-time FMRI and TMS study. , 2012 .

[27]  R W Cox,et al.  Real‐Time Functional Magnetic Resonance Imaging , 1995, Magnetic resonance in medicine.

[28]  Talma Hendler,et al.  Dual array EEG-fMRI: An approach for motion artifact suppression in EEG recorded simultaneously with fMRI , 2016, NeuroImage.

[29]  Karl J. Friston,et al.  Nonlinear Dynamic Causal Models for Fmri Nonlinear Dynamic Causal Models for Fmri Nonlinear Dynamic Causal Models for Fmri , 2022 .

[30]  João Jorge,et al.  Towards high-quality simultaneous EEG-fMRI at 7T: Detection and reduction of EEG artifacts due to head motion , 2015, NeuroImage.

[31]  Vince D. Calhoun,et al.  A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data , 2009, NeuroImage.

[32]  Seungjin Choi,et al.  A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery , 2015, Journal of Neuroscience Methods.

[33]  Mark Chiew,et al.  Development and Application of Methods for Real-time fMRI Neurofeedback , 2013 .

[34]  Nadim Joni Shah,et al.  Simultaneous EEG–fMRI acquisition at low, high and ultra-high magnetic fields up to 9.4T: Perspectives and challenges , 2014, NeuroImage.

[35]  Bettina Sorger,et al.  Real-Time Self-Regulation of Emotion Networks in Patients with Depression , 2012, PloS one.

[36]  N. Birbaumer,et al.  Self-regulation of Slow Cortical Potentials: A New Treatment for Children With Attention-Deficit/Hyperactivity Disorder , 2006, Pediatrics.

[37]  N. Turk-Browne,et al.  Optimizing real time fMRI neurofeedback for therapeutic discovery and development , 2014, NeuroImage: Clinical.

[38]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[39]  Nathan Intrator,et al.  Categorized EEG Neurofeedback Performance Unveils Simultaneous fMRI Deep Brain Activation , 2011, MLINI.

[40]  Satrajit S. Ghosh,et al.  Computing moment-to-moment BOLD activation for real-time neurofeedback , 2010, NeuroImage.

[41]  U. Strehl,et al.  Modification of Slow Cortical Potentials in Patients with Refractory Epilepsy: A Controlled Outcome Study , 2001, Epilepsia.

[42]  Richard M. Leahy,et al.  Adaptive filters for monitoring localized brain activity from surface potential time series , 1992, [1992] Conference Record of the Twenty-Sixth Asilomar Conference on Signals, Systems & Computers.

[43]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[44]  Hideki Nakano,et al.  Brain Activity during the Observation, Imagery, and Execution of Tool Use: An fNIRS/EEG Study , 2012 .

[45]  Rory A Cooper,et al.  Participatory design in the development of the wheelchair convoy system , 2008, Journal of NeuroEngineering and Rehabilitation.

[46]  Seung-Schik Yoo,et al.  Functional MRI for neurofeedback: feasibility studyon a hand motor task , 2002, Neuroreport.

[47]  Book Forum,et al.  Introduction to Quantitative Eeg and Neurofeedback , 2022 .

[48]  D. Lehmann,et al.  Functional imaging with low-resolution brain electromagnetic tomography (LORETA): a review. , 2002, Methods and findings in experimental and clinical pharmacology.

[49]  R. Turner,et al.  Characterizing Dynamic Brain Responses with fMRI: A Multivariate Approach , 1995, NeuroImage.

[50]  Epifanio Bagarinao,et al.  Estimation of general linear model coefficients for real-time application , 2003, NeuroImage.

[51]  Robert Turner,et al.  A Method for Removing Imaging Artifact from Continuous EEG Recorded during Functional MRI , 2000, NeuroImage.

[52]  S. W. Roberts.  Control chart tests based on geometric moving averages , 2000 .

[53]  Nikolaus Weiskopf,et al.  Real-time fMRI and its application to neurofeedback , 2012, NeuroImage.

[54]  Isabelle Corouge,et al.  On the feasibility and specificity of simultaneous EEG and ASL MRI at 3T , 2015 .

[55]  Cuntai Guan,et al.  A multimodal fNIRS and EEG-based BCI study on motor imagery and passive movement , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[56]  Karl J. Friston,et al.  Modelling functional integration: a comparison of structural equation and dynamic causal models , 2004, NeuroImage.

[57]  S Posse,et al.  Functional magnetic resonance imaging in real time (FIRE): Sliding‐window correlation analysis and reference‐vector optimization , 2000, Magnetic resonance in medicine.

[58]  F. Meinecke,et al.  Analysis of Multimodal Neuroimaging Data , 2011, IEEE Reviews in Biomedical Engineering.

[59]  N. Birbaumer,et al.  Learned regulation of brain metabolism , 2013, Trends in Cognitive Sciences.

[60]  Synergetic fMRI-EEG Brain Mapping in Alpha-Rhythm Voluntary Control Mode , 2015, Bulletin of Experimental Biology and Medicine.

[61]  Brain activity. , 2014, Nature nanotechnology.

[62]  Andrea Bergmann,et al.  Statistical Parametric Mapping The Analysis Of Functional Brain Images , 2016 .

[63]  Han Yuan,et al.  EEG-assisted retrospective motion correction for fMRI: E-REMCOR , 2012, NeuroImage.

[64]  Daniel Gembris,et al.  Functional Magnetic Resonance Imaging in Real-Time (FIRE) , 2000 .

[65]  João Jorge,et al.  EEG–fMRI integration for the study of human brain function , 2014, NeuroImage.

[66]  M. Fukunaga,et al.  Sources of functional magnetic resonance imaging signal fluctuations in the human brain at rest: a 7 T study. , 2009, Magnetic resonance imaging.

[67]  Kymberly D. Young,et al.  Correlation between amygdala BOLD activity and frontal EEG asymmetry during real-time fMRI neurofeedback training in patients with depression , 2014, NeuroImage: Clinical.

[68]  C. Vogel.  Computational Methods for Inverse Problems , 1987 .

[69]  Pedro A. Valdes-Sosa,et al.  Tensor Analysis and Fusion of Multimodal Brain Images , 2015, Proceedings of the IEEE.

[70]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[71]  Maarten De Vos,et al.  Real-time EEG feedback during simultaneous EEG–fMRI identifies the cortical signature of motor imagery , 2015, NeuroImage.

[72]  Jose M. Sanchez-Bornot,et al.  Model driven EEG/fMRI fusion of brain oscillations , 2009, Human brain mapping.

[73]  Tilo Kircher,et al.  Acquired self‐control of insula cortex modulates emotion recognition and brain network connectivity in schizophrenia , 2013, Human brain mapping.

[74]  Han Yuan,et al.  Self-regulation of human brain activity using simultaneous real-time fMRI and EEG neurofeedback , 2013, NeuroImage.

[75]  R. Goebel,et al.  Real-Time Functional Magnetic Resonance Imaging Neurofeedback for Treatment of Parkinson's Disease , 2011, The Journal of Neuroscience.

[76]  Dimitri Van De Ville,et al.  Connectivity-based neurofeedback: Dynamic causal modeling for real-time fMRI☆ , 2013, Neuroimage.

[77]  A. Villringer,et al.  Simultaneous EEG–fMRI , 2006, Neuroscience & Biobehavioral Reviews.

[78]  John D E Gabrieli,et al.  Control over brain activation and pain learned by using real-time functional MRI. , 2005, Proceedings of the National Academy of Sciences of the United States of America.