Feature-Based Fusion of Medical Imaging Data

The acquisition of multiple brain imaging types for a given study is a very common practice. There have been a number of approaches proposed for combining or fusing multitask or multimodal information. These can be roughly divided into those that attempt to study convergence of multimodal imaging, for example, how function and structure are related in the same region of the brain, and those that attempt to study the complementary nature of modalities, for example, utilizing temporal EEG information and spatial functional magnetic resonance imaging information. Within each of these categories, one can attempt data integration (the use of one imaging modality to improve the results of another) or true data fusion (in which multiple modalities are utilized to inform one another). We review both approaches and present a recent computational approach that first preprocesses the data to compute features of interest. The features are then analyzed in a multivariate manner using independent component analysis. We describe the approach in detail and provide examples of how it has been used for different fusion tasks. We also propose a method for selecting which combination of modalities provides the greatest value in discriminating groups. Finally, we summarize and describe future research topics.

[1]  P. Grenier,et al.  MR imaging of intravoxel incoherent motions: application to diffusion and perfusion in neurologic disorders. , 1986, Radiology.

[2]  Jean-Francois Cardoso,et al.  Source separation using higher order moments , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[3]  J. Tsuruda,et al.  Diffusion-weighted MR imaging of anisotropic water diffusion in cat central nervous system. , 1990, Radiology.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Guido Gerig,et al.  Unsupervised tissue type segmentation of 3D dual-echo MR head data , 1992, Image Vis. Comput..

[6]  L O Hall,et al.  Review of MR image segmentation techniques using pattern recognition. , 1993, Medical physics.

[7]  R. Gupta,et al.  Polynomial modeling and reduction of RF body coil spatial inhomogeneity in MRI , 1993, IEEE Trans. Medical Imaging.

[8]  P. Basser,et al.  Estimation of the effective self-diffusion tensor from the NMR spin echo. , 1994, Journal of magnetic resonance. Series B.

[9]  Karl J. Friston,et al.  Analysis of functional MRI time‐series , 1994, Human Brain Mapping.

[10]  F. Gonzalez-Lima,et al.  Structural equation modeling and its application to network analysis in functional brain imaging , 1994 .

[11]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited , 1995, NeuroImage.

[12]  C C Wood,et al.  Mapping function in the human brain with magnetoencephalography, anatomical magnetic resonance imaging, and functional magnetic resonance imaging. , 1995, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[13]  Karl J. Friston,et al.  Spatial registration and normalization of images , 1995 .

[14]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited—Again , 1995, NeuroImage.

[15]  P. Basser Inferring microstructural features and the physiological state of tissues from diffusion‐weighted images , 1995, NMR in biomedicine.

[16]  P. Basser,et al.  Microstructural and physiological features of tissues elucidated by quantitative-diffusion-tensor MRI. , 1996, Journal of magnetic resonance. Series B.

[17]  Otmar Loffeld,et al.  Information gained by data fusion , 1996, Other Conferences.

[18]  W. Eric L. Grimson,et al.  Adaptive Segmentation of MRI Data , 1995, CVRMed.

[19]  J. V. Haxby,et al.  Spatial Pattern Analysis of Functional Brain Images Using Partial Least Squares , 1996, NeuroImage.

[20]  Karl J. Friston,et al.  A multivariate analysis of PET activation studies , 1996, Human brain mapping.

[21]  P T Fox,et al.  The growth of human brain mapping , 1997, Human brain mapping.

[22]  S Makeig,et al.  Blind separation of auditory event-related brain responses into independent components. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[23]  D Le Bihan,et al.  Latencies in fMRI time‐series: effect of slice acquisition order and perception , 1997, NMR in biomedicine.

[24]  Ron Kikinis,et al.  Adaptive Template Moderated Spatially Varying Statistical Classification , 1998, MICCAI.

[25]  T. Sejnowski,et al.  Human Brain Mapping 6:368–372(1998) � Independent Component Analysis of fMRI Data: Examining the Assumptions , 2022 .

[26]  S Makeig,et al.  Analysis of fMRI data by blind separation into independent spatial components , 1998, Human brain mapping.

[27]  Alan C. Evans,et al.  Applications of random field theory to functional connectivity , 1998, Human brain mapping.

[28]  Aapo Hyvärinen,et al.  Survey on Independent Component Analysis , 1999 .

[29]  Alan C. Evans,et al.  Detecting changes in nonisotropic images , 1999, Human brain mapping.

[30]  L. K. Hansen,et al.  Plurality and Resemblance in fMRI Data Analysis , 1999, NeuroImage.

[31]  K. Kiehl,et al.  An event-related potential investigation of response inhibition in schizophrenia and psychopathy , 2000, Biological Psychiatry.

[32]  Karl J. Friston,et al.  Voxel-Based Morphometry—The Methods , 2000, NeuroImage.

[33]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[34]  Sinisa Pajevic,et al.  Color schemes to represent the orientation of anisotropic tissues from diffusion tensor data: Application to white matter fiber tract mapping in the human brain , 1999, Magnetic resonance in medicine.

[35]  A M Dale,et al.  Measuring the thickness of the human cerebral cortex from magnetic resonance images. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[36]  M S Cohen,et al.  Rapid and effective correction of RF inhomogeneity for high field magnetic resonance imaging , 2000, Human brain mapping.

[37]  V D Calhoun,et al.  Spatial and temporal independent component analysis of functional MRI data containing a pair of task‐related waveforms , 2001, Human brain mapping.

[38]  Martin Styner,et al.  Shape versus Size: Improved Understanding of the Morphology of Brain Structures , 2001, MICCAI.

[39]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[40]  S. Iversen,et al.  A neuropsychological investigation of prefrontal cortex involvement in acute mania. , 2001, The American journal of psychiatry.

[41]  R. Leahy,et al.  Magnetic Resonance Image Tissue Classification Using a Partial Volume Model , 2001, NeuroImage.

[42]  T. Sejnowski,et al.  Analysis and visualization of single‐trial event‐related potentials , 2001, Human brain mapping.

[43]  J. D. Gorman,et al.  Alpha-Divergence for Classification, Indexing and Retrieval (Revised 2) , 2002 .

[44]  Deborah A. Vitacco,et al.  Correspondence of event‐related potential tomography and functional magnetic resonance imaging during language processing , 2002, Human brain mapping.

[45]  E. Formisano,et al.  Exploring brain connectivity: a new frontier in systems neuroscience , 2002, Trends in Neurosciences.

[46]  F. Savopol,et al.  MERGING OF HETEROGENEOUS DATA FOR EMERGENCY MAPPING: DATA INTEGRATION OR DATA FUSION? , 2002 .

[47]  A. Braun,et al.  Simulating Transcranial Magnetic Stimulation during PET with a Large-Scale Neural Network Model of the Prefrontal Cortex and the Visual System , 2002, NeuroImage.

[48]  L. Younes,et al.  On the metrics and euler-lagrange equations of computational anatomy. , 2002, Annual review of biomedical engineering.

[49]  David Poeppel,et al.  How can EEG/MEG and fMRI/PET data be combined? , 2002, Human brain mapping.

[50]  J. Pekar,et al.  Different activation dynamics in multiple neural systems during simulated driving , 2002, Human brain mapping.

[51]  J. Pekar,et al.  Different activation dynamics in multiple neural systems during simulated driving , 2002, Human Brain Mapping.

[52]  L. K. Hansen,et al.  Independent component analysis of functional MRI: what is signal and what is noise? , 2003, Current Opinion in Neurobiology.

[53]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[54]  Nick C Fox,et al.  Computer-assisted imaging to assess brain structure in healthy and diseased brains , 2003, The Lancet Neurology.

[55]  Karl J. Friston,et al.  A Dynamic Causal Modeling Study on Category Effects: BottomUp or TopDown Mediation? , 2003, Journal of Cognitive Neuroscience.

[56]  Toshiharu Nakai,et al.  Application of independent component analysis to magnetic resonance imaging for enhancing the contrast of gray and white matter , 2004, NeuroImage.

[57]  C. Mervis,et al.  Neural Basis of Genetically Determined Visuospatial Construction Deficit in Williams Syndrome , 2004, Neuron.

[58]  Karsten Specht,et al.  Brain localization of attentional control in different age groups by combining functional and structural MRI , 2004, NeuroImage.

[59]  V. Calhoun,et al.  Voxel-based morphometry versus region of interest: a comparison of two methods for analyzing gray matter differences in schizophrenia , 2005, Schizophrenia Research.

[60]  V. Calhoun,et al.  ‘ UNMIXING ’ FUNCTIONAL MAGNETIC RESONANCE IMAGING WITH INDEPENDENT COMPONENT ANALYSIS , 2005 .

[61]  Godfrey Pearlson,et al.  An adaptive reflexive processing model of neurocognitive function: supporting evidence from a large scale (n = 100) fMRI study of an auditory oddball task , 2005, NeuroImage.

[62]  Anthony Randal McIntosh,et al.  Overlap in the Functional Neural Systems Involved in Semantic and Episodic Memory Retrieval , 2005, Journal of Cognitive Neuroscience.

[63]  J. Pekar,et al.  A method for multitask fMRI data fusion applied to schizophrenia , 2006, Human brain mapping.

[64]  T. Adali,et al.  Unmixing fMRI with independent component analysis , 2006, IEEE Engineering in Medicine and Biology Magazine.

[65]  J. Pekar,et al.  Method for multimodal analysis of independent source differences in schizophrenia: Combining gray matter structural and auditory oddball functional data , 2006, Human brain mapping.

[66]  Vince D. Calhoun,et al.  Neuronal chronometry of target detection: Fusion of hemodynamic and event-related potential data , 2005, NeuroImage.

[67]  V.D. Calhoun,et al.  Detection of Schizophrenia using FMRI Data via Projection Pursuit , 2007, 2007 IEEE Workshop on Machine Learning for Signal Processing.

[68]  Vince D. Calhoun,et al.  PARALLEL INDEPENDENT COMPONENT ANALYSIS FOR MULTIMODAL ANALYSIS: APPLICATION TO FMRI AND EEG DATA , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[69]  Vince D. Calhoun,et al.  A novel approach to analyzing fMRI and SNP data via parallel independent component analysis , 2007, SPIE Medical Imaging.

[70]  V. Calhoun,et al.  Joint independent component analysis for simultaneous EEG-fMRI: principle and simulation. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[71]  Vince D. Calhoun,et al.  A projection pursuit algorithm to classify individuals using fMRI data: Application to schizophrenia , 2008, NeuroImage.

[72]  R. Quiroga,et al.  Unmixing concurrent EEG-fMRI with parallel independent component analysis. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[73]  V. Calhoun,et al.  Combining fMRI and SNP data to investigate connections between brain function and genetics using parallel ICA , 2009, Human brain mapping.

[74]  Vince D. Calhoun,et al.  Feature-Based Fusion of Medical Imaging Data , 2009, IEEE Transactions on Information Technology in Biomedicine.

[75]  Karl J. Friston,et al.  Dynamic causal modeling , 2010, Scholarpedia.