Multimodal neuroimaging computing: the workflows, methods, and platforms.

The last two decades have witnessed the explosive growth in the development and use of noninvasive neuroimaging technologies that advance the research on human brain under normal and pathological conditions. Multimodal neuroimaging has become a major driver of current neuroimaging research due to the recognition of the clinical benefits of multimodal data, and the better access to hybrid devices. Multimodal neuroimaging computing is very challenging, and requires sophisticated computing to address the variations in spatiotemporal resolution and merge the biophysical/biochemical information. We review the current workflows and methods for multimodal neuroimaging computing, and also demonstrate how to conduct research using the established neuroimaging computing packages and platforms.

[1]  Siqi Liu,et al.  Multimodal neuroimaging computing: a review of the applications in neuropsychiatric disorders , 2015, Brain Informatics.

[2]  Michael W. Weiner,et al.  Widespread white matter degeneration preceding the onset of dementia , 2015, Alzheimer's & Dementia.

[3]  Natasha Lepore,et al.  Fiber estimation and tractography in diffusion MRI: Development of simulated brain images and comparison of multi-fiber analysis methods at clinical b-values , 2015, NeuroImage.

[4]  Sidong Liu,et al.  Longitudinal brain MR retrieval with diffeomorphic demons registration: What happened to those patients with similar changes? , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[5]  Jagadeesan Jayender,et al.  Multimodal imaging for improved diagnosis and treatment of cancers , 2015, Cancer.

[6]  William A. Copen,et al.  Multimodal Imaging in Acute Ischemic Stroke , 2015, Current Treatment Options in Cardiovascular Medicine.

[7]  Stefan Eberl,et al.  The relationship between neuropsychological functioning and FDG-PET hypometabolism in intractable mesial temporal lobe epilepsy , 2015, Epilepsy & Behavior.

[8]  N. Bargalló,et al.  PET/MRI and PET/MRI/SISCOM coregistration in the presurgical evaluation of refractory focal epilepsy , 2015, Epilepsy Research.

[9]  Seong-Whan Lee,et al.  Latent feature representation with stacked auto-encoder for AD/MCI diagnosis , 2013, Brain Structure and Function.

[10]  Z. Yao,et al.  A review of structural and functional brain networks: small world and atlas , 2015, Brain Informatics.

[11]  Leonardo Bonilha,et al.  Quantitative MRI in refractory temporal lobe epilepsy: relationship with surgical outcomes. , 2015, Quantitative imaging in medicine and surgery.

[12]  Gilson Vieira,et al.  Multimodal imaging of mild traumatic brain injury and persistent postconcussion syndrome , 2014, Brain and behavior.

[13]  Peter Savadjiev,et al.  Fusion of white and gray matter geometry: A framework for investigating brain development , 2014, Medical Image Anal..

[14]  Ning Yang,et al.  Fusing DTI and fMRI data: A survey of methods and applications , 2014, NeuroImage.

[15]  Seong-Whan Lee,et al.  Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis , 2014, NeuroImage.

[16]  Alan L. Yuille,et al.  Non-negative matrix factorization of multimodal MRI, fMRI and phenotypic data reveals differential changes in default mode subnetworks in ADHD , 2014, NeuroImage.

[17]  Xiaofeng Zhu,et al.  A novel matrix-similarity based loss function for joint regression and classification in AD diagnosis , 2014, NeuroImage.

[18]  Andrew Zalesky,et al.  Connectomic Disturbances in Attention-Deficit/Hyperactivity Disorder: A Whole-Brain Tractography Analysis , 2014, Biological Psychiatry.

[19]  Arno Klein,et al.  Large-scale evaluation of ANTs and FreeSurfer cortical thickness measurements , 2014, NeuroImage.

[20]  Sébastien Ourselin,et al.  Simulating Neurodegeneration through Longitudinal Population Analysis of Structural and Diffusion Weighted MRI Data , 2014, MICCAI.

[21]  Sidong Liu,et al.  Article in Press G Model Computerized Medical Imaging and Graphics Multi-channel Neurodegenerative Pattern Analysis and Its Application in Alzheimer's Disease Characterization , 2022 .

[22]  H. Benali,et al.  Brain networks disconnection in early multiple sclerosis cognitive deficits: An anatomofunctional study , 2014, Human brain mapping.

[23]  Peter Kochunov,et al.  Multimodal white matter imaging to investigate reduced fractional anisotropy and its age-related decline in schizophrenia , 2014, Psychiatry Research: Neuroimaging.

[24]  M. Phillips,et al.  A critical appraisal of neuroimaging studies of bipolar disorder: toward a new conceptualization of underlying neural circuitry and a road map for future research. , 2014, The American journal of psychiatry.

[25]  Danielle S. Bassett,et al.  Brain Network Adaptability across Task States , 2014, PLoS Comput. Biol..

[26]  Jordan W. Smoller,et al.  Dissociable Genetic Contributions to Error Processing: A Multimodal Neuroimaging Study , 2014, PloS one.

[27]  J. Gee,et al.  The Insight ToolKit image registration framework , 2014, Front. Neuroinform..

[28]  Motoaki Kawanabe,et al.  Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information , 2014, NeuroImage.

[29]  Sterling C. Johnson,et al.  Associations between white matter microstructure and amyloid burden in preclinical Alzheimer's disease: A multimodal imaging investigation , 2014, NeuroImage: Clinical.

[30]  Vinh Thai Nguyen,et al.  The superior temporal sulcus and the N170 during face processing: Single trial analysis of concurrent EEG–fMRI , 2014, NeuroImage.

[31]  James T. Patrie,et al.  Multimodal MR imaging model to predict tumor infiltration in patients with gliomas , 2014, Neuroradiology.

[32]  Martin Styner,et al.  DTIPrep: quality control of diffusion-weighted images , 2014, Front. Neuroinform..

[33]  Joaquim Radua,et al.  Multimodal voxel-based meta-analysis of structural and functional magnetic resonance imaging studies in those at elevated genetic risk of developing schizophrenia , 2014, Psychiatry Research: Neuroimaging.

[34]  C. Pozzilli,et al.  Multiple sclerosis: altered thalamic resting-state functional connectivity and its effect on cognitive function. , 2014, Radiology.

[35]  Joaquim Radua,et al.  Multimodal Voxel-Based Meta-Analysis of White Matter Abnormalities in Obsessive–Compulsive Disorder , 2014, Neuropsychopharmacology.

[36]  Dianfu Ma,et al.  Multiview Locally Linear Embedding for Effective Medical Image Retrieval , 2013, PloS one.

[37]  M. Mintun,et al.  Comparing positron emission tomography imaging and cerebrospinal fluid measurements of β‐amyloid , 2013, Annals of neurology.

[38]  N. Turk-Browne Functional Interactions as Big Data in the Human Brain , 2013, Science.

[39]  Carl-Fredrik Westin,et al.  Fiber clustering versus the parcellation-based connectome , 2013, NeuroImage.

[40]  Xiaogang Wang,et al.  Multifold Bayesian Kernelization in Alzheimer's Diagnosis , 2013, MICCAI.

[41]  Satrajit S. Ghosh,et al.  Instrumentation bias in the use and evaluation of scientific software: recommendations for reproducible practices in the computational sciences , 2013, Front. Neurosci..

[42]  Sidong Liu,et al.  A supervised multiview spectral embedding method for neuroimaging classification , 2013, 2013 IEEE International Conference on Image Processing.

[43]  C. Jack,et al.  Effectiveness of regional DTI measures in distinguishing Alzheimer's disease, MCI, and normal aging☆ , 2013, NeuroImage: Clinical.

[44]  Sidong Liu,et al.  Localized Sparse Code Gradient in Alzheimer's disease staging , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[45]  Agneta Nordberg,et al.  Amyloid tracers detect multiple binding sites in Alzheimer's disease brain tissue. , 2013, Brain : a journal of neurology.

[46]  Valerie Kirsch,et al.  Convergent Findings of Altered Functional and Structural Brain Connectivity in Individuals with High Functioning Autism: A Multimodal MRI Study , 2013, PloS one.

[47]  Thomas R. Knösche,et al.  White matter integrity, fiber count, and other fallacies: The do's and don'ts of diffusion MRI , 2013, NeuroImage.

[48]  Justin P. Haldar,et al.  Linear transforms for Fourier data on the sphere: Application to high angular resolution diffusion MRI of the brain , 2013, NeuroImage.

[49]  Sidong Liu,et al.  Multi-Channel brain atrophy pattern analysis in neuroimaging retrieval , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[50]  Joachim Gross,et al.  Good practice for conducting and reporting MEG research , 2013, NeuroImage.

[51]  Victor Alves,et al.  A hitchhiker's guide to diffusion tensor imaging , 2012, Front. Neurosci..

[52]  Milan Sonka,et al.  3D Slicer as an image computing platform for the Quantitative Imaging Network. , 2012, Magnetic resonance imaging.

[53]  Sébastien Ourselin,et al.  Cortical Folding Analysis on Patients with Alzheimer's Disease and Mild Cognitive Impairment , 2012, MICCAI.

[54]  Huiguang He,et al.  Classification of ADHD children through multimodal magnetic resonance imaging , 2012, Front. Syst. Neurosci..

[55]  Sidong Liu,et al.  Multiscale and multiorientation feature extraction with degenerative patterns for 3D neuroimaging retrieval , 2012, 2012 19th IEEE International Conference on Image Processing.

[56]  R. Coleman,et al.  Cerebral PET with florbetapir compared with neuropathology at autopsy for detection of neuritic amyloid-β plaques: a prospective cohort study , 2012, The Lancet Neurology.

[57]  Jürgen Scheins,et al.  Multimodal imaging utilising integrated MR-PET for human brain tumour assessment , 2012, European Radiology.

[58]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Peer Eysel,et al.  MRI in patients with pacemakers: overview and procedural management. , 2012, Deutsches Arzteblatt international.

[60]  Sidong Liu,et al.  A 3D difference-of-Gaussian-based lesion detector for brain PET , 2012, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[61]  Abraham Z. Snyder,et al.  Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion , 2012, NeuroImage.

[62]  Sidong Liu,et al.  Generalized regional disorder-sensitive-weighting scheme for 3D neuroimaging retrieval , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[63]  A. Toga,et al.  Comparison of acute and chronic traumatic brain injury using semi-automatic multimodal segmentation of MR volumes. , 2011, Journal of neurotrauma.

[64]  Yuan Qi,et al.  Identifying Neuroimaging and Proteomic Biomarkers for MCI and AD via the Elastic Net , 2011, MBIA.

[65]  Vince D. Calhoun,et al.  Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model , 2011, NeuroImage.

[66]  Daniel Rueckert,et al.  Automatic morphometry in Alzheimer's disease and mild cognitive impairment☆☆☆ , 2011, NeuroImage.

[67]  Sidong Liu,et al.  Localized functional neuroimaging retrieval using 3D discrete curvelet transform , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[68]  Brian B. Avants,et al.  Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge , 2011, IEEE Transactions on Medical Imaging.

[69]  Paul M. Thompson,et al.  Characterizing Alzheimer's disease using a hypometabolic convergence index , 2011, NeuroImage.

[70]  Margot J. Taylor,et al.  Review of neuroimaging in autism spectrum disorders: what have we learned and where we go from here , 2011, Molecular autism.

[71]  Daoqiang Zhang,et al.  Multimodal classification of Alzheimer's disease and mild cognitive impairment , 2011, NeuroImage.

[72]  C. McDougle,et al.  Structural and functional magnetic resonance imaging of autism spectrum disorders , 2011, Brain Research.

[73]  Vikas Singh,et al.  Predictive markers for AD in a multi-modality framework: An analysis of MCI progression in the ADNI population , 2011, NeuroImage.

[74]  Ronald Pierson,et al.  Fully automated analysis using BRAINS: AutoWorkup , 2011, NeuroImage.

[75]  D. Louis Collins,et al.  Unbiased average age-appropriate atlases for pediatric studies , 2011, NeuroImage.

[76]  Sidong Liu,et al.  3D neurological image retrieval with localized pathology-centric CMRGlc patterns , 2010, 2010 IEEE International Conference on Image Processing.

[77]  Yongdong Zhang,et al.  Multiview Spectral Embedding , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[78]  Anderson M. Winkler,et al.  Cortical thickness or grey matter volume? The importance of selecting the phenotype for imaging genetics studies , 2010, NeuroImage.

[79]  Suyash P. Awate,et al.  Cerebral cortical folding analysis with multivariate modeling and testing: Studies on gender differences and neonatal development , 2010, NeuroImage.

[80]  Maximilian F Reiser,et al.  Global trends in hybrid imaging. , 2010, Radiology.

[81]  Olaf Sporns,et al.  Complex network measures of brain connectivity: Uses and interpretations , 2010, NeuroImage.

[82]  Daniel Rueckert,et al.  Improving intersubject image registration using tissue-class information benefits robustness and accuracy of multi-atlas based anatomical segmentation , 2010, NeuroImage.

[83]  Arnaud Cachia,et al.  In-vivo measurement of cortical morphology: means and meanings. , 2010, Current opinion in neurology.

[84]  Edward T. Bullmore,et al.  Whole-brain anatomical networks: Does the choice of nodes matter? , 2010, NeuroImage.

[85]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[86]  C D Claussen,et al.  Switching on the Lights for Real-Time Multimodality Tumor Neuroimaging: The Integrated Positron-Emission Tomography/MR Imaging System , 2010, American Journal of Neuroradiology.

[87]  Peter Savadjiev,et al.  Local white matter geometry from diffusion tensor gradients , 2010, NeuroImage.

[88]  Erick Jorge Canales-Rodríguez,et al.  Medial prefrontal cortex pathology in schizophrenia as revealed by convergent findings from multimodal imaging , 2010, Molecular Psychiatry.

[89]  Lene Rosendahl,et al.  Late gadolinium uptake demonstrated with magnetic resonance in patients where automated PERFIT analysis of myocardial SPECT suggests irreversible perfusion defect , 2008, BMC Medical Imaging.

[90]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[91]  A. Fagan,et al.  Multimodal techniques for diagnosis and prognosis of Alzheimer's disease , 2009, Nature.

[92]  Alain Trouvé,et al.  Statistical models of sets of curves and surfaces based on currents , 2009, Medical Image Anal..

[93]  Vikas Singh,et al.  MKL for Robust Multi-modality AD Classification , 2009, MICCAI.

[94]  M. Pontecorvo,et al.  The use of the exploratory IND in the evaluation and development of 18F-PET radiopharmaceuticals for amyloid imaging in the brain: a review of one company's experience. , 2009, The quarterly journal of nuclear medicine and molecular imaging : official publication of the Italian Association of Nuclear Medicine (AIMN) [and] the International Association of Radiopharmacology (IAR), [and] Section of the Society of....

[95]  Daniel Rueckert,et al.  Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy , 2009, NeuroImage.

[96]  V. Libri,et al.  Interaction of the amyloid imaging tracer FDDNP with hallmark Alzheimer’s disease pathologies , 2009, Journal of neurochemistry.

[97]  Tom Vercauteren,et al.  Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.

[98]  Tianzi Jiang,et al.  Multimodal Magnetic Resonance Imaging for Brain Disorders: Advances and Perspectives , 2008, Brain Imaging and Behavior.

[99]  B. He,et al.  Multimodal Functional Neuroimaging: Integrating Functional MRI and EEG/MEG , 2008, IEEE Reviews in Biomedical Engineering.

[100]  Arthur W. Toga,et al.  Construction of a 3D probabilistic atlas of human cortical structures , 2008, NeuroImage.

[101]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[102]  Meritxell Bach Cuadra,et al.  A Surface-Based Approach to Quantify Local Cortical Gyrification , 2008, IEEE Transactions on Medical Imaging.

[103]  Abraham Z. Snyder,et al.  A default mode of brain function: A brief history of an evolving idea , 2007, NeuroImage.

[104]  Alan Connelly,et al.  Robust determination of the fibre orientation distribution in diffusion MRI: Non-negativity constrained super-resolved spherical deconvolution , 2007, NeuroImage.

[105]  Andrea Cherubini,et al.  Multimodal fMRI tractography in normal subjects and in clinically recovered traumatic brain injury patients , 2007, NeuroImage.

[106]  Daniel Rueckert,et al.  Automatic anatomical brain MRI segmentation combining label propagation and decision fusion , 2006, NeuroImage.

[107]  Lei Ding,et al.  Integration of EEG/MEG with MRI and fMRI , 2006, IEEE Engineering in Medicine and Biology Magazine.

[108]  S. Taulu,et al.  Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements , 2006, Physics in medicine and biology.

[109]  William E. Lorensen,et al.  The NA-MIC Kit: ITK, VTK, pipelines, grids and 3D slicer as an open platform for the medical image computing community , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[110]  P. Hagmann,et al.  Mapping complex tissue architecture with diffusion spectrum magnetic resonance imaging , 2005, Magnetic resonance in medicine.

[111]  Osama Mawlawi,et al.  PET/CT imaging artifacts. , 2005, Journal of nuclear medicine technology.

[112]  M. Miller,et al.  Computational anatomy and neuropsychiatric disease: probabilistic assessment of variation and statistical inference of group difference, hemispheric asymmetry, and time-dependent change , 2004, NeuroImage.

[113]  D. Tuch Q‐ball imaging , 2004, Magnetic resonance in medicine.

[114]  Masako Okamoto,et al.  Multimodal assessment of cortical activation during apple peeling by NIRS and fMRI , 2004, NeuroImage.

[115]  Timothy Edward John Behrens,et al.  Characterization and propagation of uncertainty in diffusion‐weighted MR imaging , 2003, Magnetic resonance in medicine.

[116]  Alexander Hammers,et al.  Three‐dimensional maximum probability atlas of the human brain, with particular reference to the temporal lobe , 2003, Human brain mapping.

[117]  A Achiron,et al.  Cognitive impairment in probable multiple sclerosis , 2003, Journal of neurology, neurosurgery, and psychiatry.

[118]  Susumu Mori,et al.  Fiber tracking: principles and strategies – a technical review , 2002, NMR in biomedicine.

[119]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[120]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[121]  J Mazziotta,et al.  A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[122]  D W Townsend,et al.  A combined PET/CT scanner: the choices. , 2001, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[123]  A M Dale,et al.  Measuring the thickness of the human cerebral cortex from magnetic resonance images. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[124]  Paul Kinahan,et al.  A combined PET/CT scanner for clinical oncology. , 2000, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[125]  Karl J. Friston,et al.  Voxel-Based Morphometry—The Methods , 2000, NeuroImage.

[126]  David Dagan Feng,et al.  Content-based retrieval of dynamic PET functional images , 2000, IEEE Transactions on Information Technology in Biomedicine.

[127]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[128]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[129]  R. Ilmoniemi,et al.  Signal-space projection method for separating MEG or EEG into components , 1997, Medical and Biological Engineering and Computing.

[130]  Karl J. Friston,et al.  Movement‐Related effects in fMRI time‐series , 1996, Magnetic resonance in medicine.

[131]  B. Biswal,et al.  Functional connectivity in the motor cortex of resting human brain using echo‐planar mri , 1995, Magnetic resonance in medicine.

[132]  R. Koeppe,et al.  A diagnostic approach in Alzheimer's disease using three-dimensional stereotactic surface projections of fluorine-18-FDG PET. , 1995, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[133]  A. Galaburda,et al.  Human Cerebral Cortex: Localization, Parcellation, and Morphometry with Magnetic Resonance Imaging , 1992, Journal of Cognitive Neuroscience.

[134]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[135]  Sidong Liu,et al.  Multimodal neuroimaging computing: a review of the applications in neuropsychiatric disorders , 2015, Brain Informatics.

[136]  Siqi Liu,et al.  Multi-Modal Neuroimaging Feature Learning for Multi-Class Diagnosis of Alzheimer’s Disease , 2015 .

[137]  Xin Yu,et al.  A combined DTI and structural MRI study in medicated-naïve chronic schizophrenia. , 2014, Magnetic Resonance Imaging.

[138]  Kaspar Anton Schindler,et al.  Neuroimaging of Epilepsy: Lesions, Networks, Oscillations , 2014, Clinical Neuroradiology.

[139]  Kirby G. Vosburgh,et al.  3D Slicer: A Platform for Subject-Specific Image Analysis, Visualization, and Clinical Support , 2014 .

[140]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[141]  M. Götte,et al.  Magnetic resonance imaging, pacemakers and implantable cardioverter-defibrillators: current situation and clinical perspective. , 2010, Netherlands heart journal : monthly journal of the Netherlands Society of Cardiology and the Netherlands Heart Foundation.

[142]  Daniel L. Rubin,et al.  Annotation and Image Markup: Accessing and Interoperating with the Semantic Content in Medical Imaging , 2009, IEEE Intelligent Systems.

[143]  John Clark,et al.  Prototype System for Semantic Retrieval of Neurological PET Images , 2007, MIMI.

[144]  Karl J. Friston Introduction Experimental design and Statistical Parametric Mapping , 2003 .

[145]  Michael J Ackerman,et al.  Engineering and algorithm design for an image processing Api: a technical report on ITK--the Insight Toolkit. , 2002, Studies in health technology and informatics.