Multi-task neural networks for joint hippocampus segmentation and clinical score regression

Feature representations extracted from hippocampus in magnetic resonance (MR) images are widely used in computer-aided Alzheimer’s disease (AD) diagnosis, and thus accurate segmentation for the hippocampus has been remaining an active research topic. Previous studies for hippocampus segmentation require either human annotation which is tedious and error-prone or pre-processing MR images via time-consuming non-linear registration. Although many automatic segmentation approaches have been proposed, their performance is often limited by the small size of hippocampus and complex confounding information around the hippocampus. In particular, human-engineered features extracted from segmented hippocampus regions (e.g., the volume of the hippocampus) are essential for brain disease diagnosis, while these features are independent of diagnosis models, leading to sub-optimal performance. To address these issues, we propose a multi-task deep learning (MDL) method for joint hippocampus segmentation and clinical score regression using MR images. The prominent advantages of our MDL method lie on that we don’t need any time-consuming non-linear registration for pre-processing MR images, and features generated by MDL are consistent with subsequent diagnosis models. Specifically, we first align all MR images onto a standard template, followed by a patch extraction process to approximately locate hippocampus regions in the template space. Using image patches as input data, we develop a multi-task convolutional neural network (CNN) for joint hippocampus segmentation and clinical score regression. The proposed CNN network contains two subnetworks, including 1) a U-Net with a Dice-like loss function for hippocampus segmentation, and 2) a convolutional neural network with a mean squared loss function for clinical regression. Note that these two subnetworks share a part of network parameters, to exploit the inherent association between these two tasks. We evaluate the proposed method on 407 subjects with MRI data from baseline Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. The experimental results suggest that our MDL method achieves promising results in both tasks of hippocampus segmentation and clinical score regression, compared with several state-of-the-art methods.

[1]  Dinggang Shen,et al.  Early Diagnosis of Alzheimer's Disease by Joint Feature Selection and Classification on Temporally Structured Support Vector Machine , 2016, MICCAI.

[2]  Alan C. Evans,et al.  A nonparametric method for automatic correction of intensity nonuniformity in MRI data , 1998, IEEE Transactions on Medical Imaging.

[3]  Lin Xie,et al.  Increased hippocampal neurogenesis in Alzheimer's disease , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[4]  W. Eric L. Grimson,et al.  A Hierarchical Algorithm for MR Brain Image Parcellation , 2007, IEEE Transactions on Medical Imaging.

[5]  Arthur W. Toga,et al.  Impact of acquisition protocols and processing streams on tissue segmentation of T1 weighted MR images , 2006, NeuroImage.

[6]  Dinggang Shen,et al.  A novel relational regularization feature selection method for joint regression and classification in AD diagnosis , 2017, Medical Image Anal..

[7]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[8]  Nicos Maglaveras,et al.  Accurate and Fully Automatic Hippocampus Segmentation Using Subject-Specific 3D Optimal Local Maps Into a Hybrid Active Contour Model , 2014, IEEE Journal of Translational Engineering in Health and Medicine.

[9]  Timothy F. Cootes,et al.  Fully Automatic Segmentation of the Proximal Femur Using Random Forest Regression Voting , 2013, IEEE Transactions on Medical Imaging.

[10]  Yaozong Gao,et al.  Structured Learning for 3-D Perivascular Space Segmentation Using Vascular Features , 2017, IEEE Transactions on Biomedical Engineering.

[11]  Yaozong Gao,et al.  Landmark-Based Alzheimer's Disease Diagnosis Using Longitudinal Structural MR Images , 2016, MCV/BAMBI@MICCAI.

[12]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[13]  Paul M. Thompson,et al.  Atlas-based hippocampus segmentation in Alzheimer's disease and mild cognitive impairment , 2005, NeuroImage.

[14]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[15]  Daoqiang Zhang,et al.  Joint Binary Classifier Learning for ECOC-Based Multi-Class Classification , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[17]  刘明霞 View-centralized multi-atlas classification for Alzheimer's disease diagnosis , 2015 .

[18]  Andrea Chincarini,et al.  Local MRI analysis approach in the diagnosis of early and prodromal Alzheimer's disease , 2011, NeuroImage.

[19]  Tianzi Jiang,et al.  Local label learning (LLL) for subcortical structure segmentation: Application to hippocampus segmentation , 2014, Human brain mapping.

[20]  Márcio Sarroglia Pinho,et al.  Automated Methods for Hippocampus Segmentation: the Evolution and a Review of the State of the Art , 2014, Neuroinformatics.

[21]  Yaozong Gao,et al.  Learning-Based Multimodal Image Registration for Prostate Cancer Radiation Therapy , 2016, MICCAI.

[22]  Yaozong Gao,et al.  Dual‐core steered non‐rigid registration for multi‐modal images via bi‐directional image synthesis , 2017, Medical Image Anal..

[23]  A. Dagher,et al.  The role of the striatum and hippocampus in planning: a PET activation study in Parkinson's disease. , 2001, Brain : a journal of neurology.

[24]  Dinggang Shen,et al.  Reveal Consistent Spatial-Temporal Patterns from Dynamic Functional Connectivity for Autism Spectrum Disorder Identification , 2016, MICCAI.

[25]  Nick C Fox,et al.  Automatic calculation of hippocampal atrophy rates using a hippocampal template and the boundary shift integral , 2007, Neurobiology of Aging.

[26]  Daniel Rueckert,et al.  Automatic anatomical brain MRI segmentation combining label propagation and decision fusion , 2006, NeuroImage.

[27]  A. Nappi,et al.  Alzheimer ' s Disease : Cell-Specific Pathology Isolates the Hippocampal Formation , 2022 .

[28]  Sang Won Seo,et al.  Fully-automated approach to hippocampus segmentation using a graph-cuts algorithm combined with atlas-based segmentation and morphological opening. , 2013, Magnetic resonance imaging.

[29]  Nick C Fox,et al.  The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods , 2008, Journal of magnetic resonance imaging : JMRI.

[30]  Xiaofeng Zhu,et al.  A novel matrix-similarity based loss function for joint regression and classification in AD diagnosis , 2014, NeuroImage.

[31]  Mert R. Sabuncu,et al.  Multi-atlas segmentation of biomedical images: A survey , 2014, Medical Image Anal..

[32]  Thierry Denoeux,et al.  Selecting radiomic features from FDG-PET images for cancer treatment outcome prediction , 2016, Medical Image Anal..

[33]  Yaozong Gao,et al.  Automatic Craniomaxillofacial Landmark Digitization via Segmentation-Guided Partially-Joint Regression Forest Model and Multiscale Statistical Features , 2016, IEEE Transactions on Biomedical Engineering.

[34]  Ron Kikinis,et al.  Statistical validation of image segmentation quality based on a spatial overlap index. , 2004, Academic radiology.

[35]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[36]  D. Louis Collins,et al.  Simultaneous segmentation and grading of anatomical structures for patient's classification: Application to Alzheimer's disease , 2012, NeuroImage.

[37]  Dinggang Shen,et al.  View‐aligned hypergraph learning for Alzheimer's disease diagnosis with incomplete multi‐modality data , 2017, Medical Image Anal..

[38]  Dinggang Shen,et al.  Landmark‐based deep multi‐instance learning for brain disease diagnosis , 2018, Medical Image Anal..

[39]  Jun Zhang,et al.  Detecting Anatomical Landmarks From Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks , 2017, IEEE Transactions on Image Processing.

[40]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[41]  Thierry Denoeux,et al.  An evidential classifier based on feature selection and two-step classification strategy , 2015, Pattern Recognit..

[42]  D. Louis Collins,et al.  Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation , 2011, NeuroImage.

[43]  Azar Zandifar,et al.  A comparison of accurate automatic hippocampal segmentation methods , 2017, NeuroImage.

[44]  Ludovico Minati,et al.  Diagnostic differentiation of mild cognitive impairment due to Alzheimer's disease using a hippocampus‐dependent test of spatial memory , 2015, Hippocampus.

[45]  Jun Zhang,et al.  Local Energy Pattern for Texture Classification Using Self-Adaptive Quantization Thresholds , 2013, IEEE Transactions on Image Processing.

[46]  Chokri Ben Amar,et al.  Alzheimer's disease diagnosis on structural MR images using circular harmonic functions descriptors on hippocampus and posterior cingulate cortex , 2015, Comput. Medical Imaging Graph..

[47]  C. Jack,et al.  MR‐based hippocampal volumetry in the diagnosis of Alzheimer's disease , 1992, Neurology.

[48]  Xiaofeng Zhu,et al.  A Novel Dynamic Hyper-graph Inference Framework for Computer Assisted Diagnosis of Neuro-Diseases , 2017, IPMI.

[49]  E. Tulving,et al.  Episodic and declarative memory: Role of the hippocampus , 1998, Hippocampus.