Weakly Supervised Deep Learning for Brain Disease Prognosis Using MRI and Incomplete Clinical Scores

As a hot topic in brain disease prognosis, predicting clinical measures of subjects based on brain magnetic resonance imaging (MRI) data helps to assess the stage of pathology and predict future development of the disease. Due to incomplete clinical labels/scores, previous learning-based studies often simply discard subjects without ground-truth scores. This would result in limited training data for learning reliable and robust models. Also, existing methods focus only on using hand-crafted features (e.g., image intensity or tissue volume) of MRI data, and these features may not be well coordinated with prediction models. In this paper, we propose a weakly supervised densely connected neural network (wiseDNN) for brain disease prognosis using baseline MRI data and incomplete clinical scores. Specifically, we first extract multiscale image patches (located by anatomical landmarks) from MRI to capture local-to-global structural information of images, and then develop a weakly supervised densely connected network for task-oriented extraction of imaging features and joint prediction of multiple clinical measures. A weighted loss function is further employed to make full use of all available subjects (even those without ground-truth scores at certain time-points) for network training. The experimental results on 1469 subjects from both ADNI-1 and ADNI-2 datasets demonstrate that our proposed method can efficiently predict future clinical measures of subjects.

[1]  T. Crow,et al.  Regional deficits in brain volume in schizophrenia: a meta-analysis of voxel-based morphometry studies. , 2005, The American journal of psychiatry.

[2]  Michael I. Jordan,et al.  Unsupervised Domain Adaptation with Residual Transfer Networks , 2016, NIPS.

[3]  Nick C Fox,et al.  Mapping the evolution of regional atrophy in Alzheimer's disease: Unbiased analysis of fluid-registered serial MRI , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[4]  A M Dale,et al.  Measuring the thickness of the human cerebral cortex from magnetic resonance images. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Dinggang Shen,et al.  COMPARE: Classification of Morphological Patterns Using Adaptive Regional Elements , 2007, IEEE Transactions on Medical Imaging.

[6]  Nick C Fox,et al.  The clinical use of structural MRI in Alzheimer disease , 2010, Nature Reviews Neurology.

[7]  Dinggang Shen,et al.  Hierarchical Fully Convolutional Network for Joint Atrophy Localization and Alzheimer's Disease Diagnosis Using Structural MRI , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Yaozong Gao,et al.  Detecting Anatomical Landmarks for Fast Alzheimer’s Disease Diagnosis , 2016, IEEE Transactions on Medical Imaging.

[9]  Dinggang Shen,et al.  Synthesizing Missing PET from MRI with Cycle-consistent Generative Adversarial Networks for Alzheimer's Disease Diagnosis , 2018, MICCAI.

[10]  Dong Ni,et al.  Relational-Regularized Discriminative Sparse Learning for Alzheimer’s Disease Diagnosis , 2017, IEEE Transactions on Cybernetics.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[13]  Alan C. Evans,et al.  A nonparametric method for automatic correction of intensity nonuniformity in MRI data , 1998, IEEE Transactions on Medical Imaging.

[15]  Daoqiang Zhang,et al.  Relationship Induced Multi-Template Learning for Diagnosis of Alzheimer’s Disease and Mild Cognitive Impairment , 2016, IEEE Transactions on Medical Imaging.

[16]  C. Jack,et al.  Serial PIB and MRI in normal, mild cognitive impairment and Alzheimer's disease: implications for sequence of pathological events in Alzheimer's disease , 2009, Brain : a journal of neurology.

[17]  J. Mazziotta,et al.  Cerebral metabolic and cognitive decline in persons at genetic risk for Alzheimer's disease. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Daoqiang Zhang,et al.  Sub-Network Kernels for Measuring Similarity of Brain Connectivity Networks in Disease Diagnosis , 2018, IEEE Transactions on Image Processing.

[19]  Paul M. Thompson,et al.  Bi-level multi-source learning for heterogeneous block-wise missing data , 2014, NeuroImage.

[20]  Daoqiang Zhang,et al.  Hierarchical fusion of features and classifier decisions for Alzheimer's disease diagnosis , 2014, Human brain mapping.

[21]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[22]  Jenny Benois-Pineau,et al.  3D CNN-based classification using sMRI and MD-DTI images for Alzheimer disease studies , 2018, ArXiv.

[23]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[24]  J. Baron,et al.  In Vivo Mapping of Gray Matter Loss with Voxel-Based Morphometry in Mild Alzheimer's Disease , 2001, NeuroImage.

[25]  Daoqiang Zhang,et al.  Sparsity Score: a Novel Graph-Preserving Feature Selection Method , 2014, Int. J. Pattern Recognit. Artif. Intell..

[26]  D. Collins,et al.  Scoring by nonlocal image patch estimator for early detection of Alzheimer's disease☆ , 2012, NeuroImage: Clinical.

[27]  Dinggang Shen,et al.  Joint Classification and Regression via Deep Multi-Task Multi-Channel Learning for Alzheimer's Disease Diagnosis , 2019, IEEE Transactions on Biomedical Engineering.

[28]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  C. Jack,et al.  Prediction of AD with MRI-based hippocampal volume in mild cognitive impairment , 1999, Neurology.

[30]  Nick C Fox,et al.  The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods , 2008, Journal of magnetic resonance imaging : JMRI.

[31]  Daoqiang Zhang,et al.  Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer's disease , 2012, NeuroImage.

[32]  Ronald Killiany,et al.  Structural Magnetic Resonance Imaging in Established and Prodromal Alzheimer Disease: A Review , 2003, Alzheimer disease and associated disorders.

[33]  H. Yamasue,et al.  Voxel-based analysis of MRI reveals anterior cingulate gray-matter volume reduction in posttraumatic stress disorder due to terrorism , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Norbert Schuff,et al.  Locally linear embedding (LLE) for MRI based Alzheimer's disease classification , 2013, NeuroImage.

[35]  Mohamad Habes,et al.  Deep Ordinal Ranking for Multi-Category Diagnosis of Alzheimer's Disease using Hippocampal MRI data , 2017, ArXiv.

[36]  Jie Zhang,et al.  Multi-task Dictionary Learning based Convolutional Neural Network for Computer aided Diagnosis with Longitudinal Images , 2017, ArXiv.

[37]  C. Jack,et al.  MR‐based hippocampal volumetry in the diagnosis of Alzheimer's disease , 1992, Neurology.

[38]  Jyrki Lötjönen,et al.  Fast and robust extraction of hippocampus from MR images for diagnostics of Alzheimer's disease , 2011, NeuroImage.

[39]  Xiaofeng Zhu,et al.  A novel matrix-similarity based loss function for joint regression and classification in AD diagnosis , 2014, NeuroImage.

[40]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[41]  Yaozong Gao,et al.  Alzheimer's Disease Diagnosis Using Landmark-Based Features From Longitudinal Structural MR Images , 2017, IEEE Journal of Biomedical and Health Informatics.

[42]  Gabriela Csurka,et al.  Domain Adaptation for Visual Applications: A Comprehensive Survey , 2017, ArXiv.

[43]  Alan C. Evans,et al.  Enhancement of MR Images Using Registration for Signal Averaging , 1998, Journal of Computer Assisted Tomography.

[44]  Richard S. J. Frackowiak,et al.  Navigation-related structural change in the hippocampi of taxi drivers. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Jun Zhang,et al.  Local Energy Pattern for Texture Classification Using Self-Adaptive Quantization Thresholds , 2013, IEEE Transactions on Image Processing.

[46]  Ghassem Tofighi,et al.  DeepAD: Alzheimer’s Disease Classification via Deep Convolutional Neural Networks using MRI and fMRI , 2016, bioRxiv.

[47]  Marie Chupin,et al.  Automatic classi fi cation of patients with Alzheimer ' s disease from structural MRI : A comparison of ten methods using the ADNI database , 2010 .

[48]  Nick C Fox,et al.  Presymptomatic hippocampal atrophy in Alzheimer's disease. A longitudinal MRI study. , 1996, Brain : a journal of neurology.

[49]  Daoqiang Zhang,et al.  Ensemble sparse classification of Alzheimer's disease , 2012, NeuroImage.

[50]  Paul M. Thompson,et al.  Multi-source feature learning for joint analysis of incomplete multiple heterogeneous neuroimaging data , 2012, NeuroImage.

[51]  Dinggang Shen,et al.  Landmark‐based deep multi‐instance learning for brain disease diagnosis , 2018, Medical Image Anal..

[52]  Xuelong Li,et al.  Modeling Disease Progression via Multisource Multitask Learners: A Case Study With Alzheimer’s Disease , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[53]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[54]  R. Gur,et al.  Unaffected Family Members and Schizophrenia Patients Share Brain Structure Patterns: A High-Dimensional Pattern Classification Study , 2008, Biological Psychiatry.

[55]  Dinggang Shen,et al.  CLASSIC: Consistent Longitudinal Alignment and Segmentation for Serial Image Computing , 2005, IPMI.

[56]  Karl J. Friston,et al.  Voxel-Based Morphometry—The Methods , 2000, NeuroImage.

[57]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.