Integration of Multivariate Data Streams With Bandpower Signals

The urge to further our understanding of multimodal neural data has recently become an important topic due to the ever increasing availability of simultaneously recorded data from different neural imaging modalities. In case where EEG is one of the modalities, it is of interest to relate a nonlinear function of the raw EEG time-domain signal, say, EEG band power, to another modality such as the hemodynamic response, as measured with NIRS or fMRI. In this work we tackle exactly this problem defining a novel algorithm that we denote multimodal source power correlation analysis (mSPoC). The validity and high performance of the mSPoC framework is demonstrated for simulated and real-world multimodal data.

[1]  A. Eke,et al.  The modified Beer–Lambert law revisited , 2006, Physics in medicine and biology.

[2]  Jérémie Mattout,et al.  Symmetrical event-related EEG/fMRI information fusion in a variational Bayesian framework , 2007, NeuroImage.

[3]  A. Dale,et al.  Improved Localizadon of Cortical Activity by Combining EEG and MEG with MRI Cortical Surface Reconstruction: A Linear Approach , 1993, Journal of Cognitive Neuroscience.

[4]  Karl J. Friston,et al.  EEG-fMRI integration: a critical review of biophysical modeling and data analysis approaches. , 2010, Journal of integrative neuroscience.

[5]  Fumikazu Miwakeichi,et al.  Concurrent EEG/fMRI analysis by multiway Partial Least Squares , 2004, NeuroImage.

[6]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[7]  S. Laughlin,et al.  An Energy Budget for Signaling in the Grey Matter of the Brain , 2001, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[8]  S. Ogawa,et al.  Oxygenation‐sensitive contrast in magnetic resonance image of rodent brain at high magnetic fields , 1990, Magnetic resonance in medicine.

[9]  Shotaro Akaho,et al.  MICA: multimodal independent component analysis , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[10]  Hellmuth Obrig,et al.  Correlates of alpha rhythm in functional magnetic resonance imaging and near infrared spectroscopy , 2003, NeuroImage.

[11]  Thom F. Oostendorp,et al.  Towards a model-based integration of co-registered electroencephalography/functional magnetic resonance imaging data with realistic neural population meshes , 2011, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[12]  Klaus-Robert Müller,et al.  Relationship between neural and hemodynamic signals during spontaneous activity studied with temporal kernel CCA. , 2010, Magnetic resonance imaging.

[13]  Klaus-Robert Müller,et al.  Improved decoding of neural activity from fMRI signals using non-separable spatiotemporal deconvolutions , 2012, NeuroImage.

[14]  Richard M. Leahy,et al.  Electromagnetic brain mapping , 2001, IEEE Signal Process. Mag..

[15]  Karl J. Friston Modalities, Modes, and Models in Functional Neuroimaging , 2009, Science.

[16]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[17]  F. Jöbsis Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. , 1977, Science.

[18]  J Riera,et al.  Fusing EEG and fMRI based on a bottom-up model: inferring activation and effective connectivity in neural masses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[19]  V. Calhoun,et al.  Joint independent component analysis for simultaneous EEG-fMRI: principle and simulation. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[20]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[21]  A. Villringer,et al.  Rolandic alpha and beta EEG rhythms' strengths are inversely related to fMRI‐BOLD signal in primary somatosensory and motor cortex , 2009, Human brain mapping.

[22]  Bin He,et al.  fMRI–EEG integrated cortical source imaging by use of time-variant spatial constraints , 2008, NeuroImage.

[23]  Tom Eichele,et al.  Removal of MRI Artifacts from EEG Recordings , 2010 .

[24]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[25]  T. Ono,et al.  Brain Cortical Mapping by Simultaneous Recording of Functional Near Infrared Spectroscopy and Electroencephalograms from the Whole Brain During Right Median Nerve Stimulation , 2009, Brain Topography.

[26]  Lars Kai Hansen,et al.  Shift-invariant multilinear decomposition of neuroimaging data , 2008, NeuroImage.

[27]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[28]  Jean-Francois Cardoso,et al.  THE THREE EASY ROUTES TO INDEPENDENT COMPONENT ANALYSIS; CONTRASTS AND GEOMETRY , 2001 .

[29]  Hellmuth Obrig,et al.  Somatosensory activation of two fingers can be discriminated with ultrahigh-density diffuse optical tomography , 2012, NeuroImage.

[30]  A K Liu,et al.  Spatiotemporal imaging of human brain activity using functional MRI constrained magnetoencephalography data: Monte Carlo simulations. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[31]  K. Uğurbil,et al.  The Spatial Dependence of the Poststimulus Undershoot as Revealed by High-Resolution BOLD- and CBV-Weighted fMRI , 2005, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[32]  H. Berger Über das Elektrenkephalogramm des Menschen , 1933, Archiv für Psychiatrie und Nervenkrankheiten.

[33]  Vince D. Calhoun,et al.  Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI , 2010, NeuroImage.

[34]  F. Meinecke,et al.  Analysis of Multimodal Neuroimaging Data , 2011, IEEE Reviews in Biomedical Engineering.

[35]  H. Berger Über das Elektrenkephalogramm des Menschen , 1929, Archiv für Psychiatrie und Nervenkrankheiten.

[36]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[37]  E. Halgren,et al.  Dynamic Statistical Parametric Mapping Combining fMRI and MEG for High-Resolution Imaging of Cortical Activity , 2000, Neuron.

[38]  Karl J. Friston,et al.  Studying spontaneous EEG activity with fMRI , 2003, Brain Research Reviews.

[39]  Klaus-Robert Müller,et al.  Temporal kernel CCA and its application in multimodal neuronal data analysis , 2010, Machine Learning.

[40]  Dirk Ostwald,et al.  An information theoretic approach to EEG–fMRI integration of visually evoked responses , 2010, NeuroImage.

[41]  Lucas C. Parra,et al.  Recipes for the linear analysis of EEG , 2005, NeuroImage.

[42]  D. Kleinfeld,et al.  Correlations of Neuronal and Microvascular Densities in Murine Cortex Revealed by Direct Counting and Colocalization of Nuclei and Vessels , 2009, The Journal of Neuroscience.

[43]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[44]  E. Hamel Perivascular nerves and the regulation of cerebrovascular tone. , 2006, Journal of applied physiology.

[45]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[46]  Jieping Ye,et al.  Finite Domain Constraint Solver Learning , 2009, IJCAI.

[47]  Mark D'Esposito,et al.  Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses , 2004, NeuroImage.

[48]  Jose M. Sanchez-Bornot,et al.  Model driven EEG/fMRI fusion of brain oscillations , 2009, Human brain mapping.

[49]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[50]  R D Pascual-Marqui,et al.  Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. , 2002, Methods and findings in experimental and clinical pharmacology.

[51]  Klaus-Robert Müller,et al.  Introduction to machine learning for brain imaging , 2011, NeuroImage.

[52]  Ann-Christine Ehlis,et al.  Enhancement of activity of the primary visual cortex during processing of emotional stimuli as measured with event‐related functional near‐infrared spectroscopy and event‐related potentials , 2008, Human brain mapping.

[53]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[54]  S. Arridge,et al.  Estimation of optical pathlength through tissue from direct time of flight measurement , 1988 .

[55]  J. Schunk,et al.  Neuroenergetics Original Research Article , 2022 .

[56]  J. Fermaglich Electric Fields of the Brain: The Neurophysics of EEG , 1982 .

[57]  Vince D. Calhoun,et al.  A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data , 2009, NeuroImage.

[58]  Stefan Haufe,et al.  Large-scale EEG/MEG source localization with spatial flexibility , 2011, NeuroImage.

[59]  J. Mayhew,et al.  Fine detail of neurovascular coupling revealed by spatiotemporal analysis of the hemodynamic response to single whisker stimulation in rat barrel cortex. , 2008, Journal of neurophysiology.

[60]  Hellmuth Obrig,et al.  Stimulus-Induced and State-Dependent Sustained Gamma Activity Is Tightly Coupled to the Hemodynamic Response in Humans , 2009, The Journal of Neuroscience.

[61]  Andrzej Cichocki,et al.  Nonnegative Tensor Factorization for Continuous EEG Classification , 2007, Int. J. Neural Syst..

[62]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..