Multi-target Interactive Neural Network for Automated Segmentation of the Hippocampus in Magnetic Resonance Imaging

The hippocampus has been recognized as an important biomarker for the diagnosis and assessment of neurological diseases. Convenient and accurate automated segmentation of the hippocampus facilitates the analysis of large-scale neuroimaging studies. This work describes a novel technique for hippocampus segmentation in magnetic resonance images, in which interactive neural network (Inter-Net) is based on 3D convolutional operations. Inter-Net achieves the interaction through two aspects: one is the compartments, which builds an exponential ensemble network that integrates numerous short networks together when forward propagation. The other is the pathways, which realizes inter-connection between feature extraction and restoration. In addition, a multi-target architecture is proposed by designing multiple objective functions in terms of evaluation index, information theory, and data distribution. The proposed architecture is validated in fivefold cross-validation on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, where the mean Dice similarity indices of 0.919 (± 0.023) and precision of 0.926 (± 0.032) for the hippocampus segmentation. The running time is approximately 42.1 s from reading the image to outputting the segmentation result in our computer configuration. We compare the experimental results of a variety of methods to prove the effectiveness of the Inter-Net and contrast integrated architectures with different objective functions to illustrate the robustness of the fusion. The proposed framework is general and can be easily extended to numerous tissue segmentation tasks while it is tailored for the hippocampus.

[1]  Meritxell Bach Cuadra,et al.  A review of atlas-based segmentation for magnetic resonance brain images , 2011, Comput. Methods Programs Biomed..

[2]  Fan Zhao,et al.  Compressing and Accelerating Neural Network for Facial Point Localization , 2018, Cognitive Computation.

[3]  Juha Koikkalainen,et al.  Fast and robust multi-atlas segmentation of brain magnetic resonance images , 2010, NeuroImage.

[4]  Hayit Greenspan,et al.  Multi-view longitudinal CNN for multiple sclerosis lesion segmentation , 2017, Eng. Appl. Artif. Intell..

[5]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[6]  Dario Pompili,et al.  Comparative performance evaluation of automated segmentation methods of hippocampus from magnetic resonance images of temporal lobe epilepsy patients. , 2016, Medical physics.

[7]  A. Dale,et al.  Whole Brain Segmentation Automated Labeling of Neuroanatomical Structures in the Human Brain , 2002, Neuron.

[8]  Anvi Vora,et al.  Relapse duration, treatment intensity, and brain tissue loss in schizophrenia: a prospective longitudinal MRI study. , 2013, The American journal of psychiatry.

[9]  Paul M. Thompson,et al.  Atlas-based hippocampus segmentation in Alzheimer's disease and mild cognitive impairment , 2005, NeuroImage.

[10]  Bruce Fischl,et al.  FreeSurfer , 2012, NeuroImage.

[11]  L. Squire,et al.  Working memory, long-term memory, and medial temporal lobe function. , 2011, Learning & memory.

[12]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[13]  H Soltanian-Zadeh,et al.  A 3D deformable surface model for segmentation of objects from volumetric data in medical images. , 1998, Computers in biology and medicine.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[16]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[17]  Dominique Hasboun,et al.  Anatomically constrained region deformation for the automated segmentation of the hippocampus and the amygdala: Method and validation on controls and patients with Alzheimer’s disease , 2007, NeuroImage.

[18]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[19]  Stefan Klein,et al.  Automated Brain Structure Segmentation Based on Atlas Registration and Appearance Models , 2012, IEEE Transactions on Medical Imaging.

[20]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[21]  Guixia Kang,et al.  3D multi-view convolutional neural networks for lung nodule classification , 2017, PloS one.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Wiro J. Niessen,et al.  Structural and diffusion MRI measures of the hippocampus and memory performance , 2012, NeuroImage.

[24]  Mert R. Sabuncu,et al.  A Generative Model for Image Segmentation Based on Label Fusion , 2010, IEEE Transactions on Medical Imaging.

[25]  Fenglong Ma,et al.  MuVAN: A Multi-view Attention Network for Multivariate Temporal Data , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[26]  Serge J. Belongie,et al.  Residual Networks are Exponential Ensembles of Relatively Shallow Networks , 2016, ArXiv.

[27]  J. Wegiel,et al.  Neurofibrillary pathology — correlation with hippocampal formation atrophy in Alzheimer disease , 1996, Neurobiology of Aging.

[28]  M. Mallar Chakravarty,et al.  Multi-atlas segmentation of the whole hippocampus and subfields using multiple automatically generated templates , 2014, NeuroImage.

[29]  Weifeng Liu,et al.  Multiview Hessian Regularization for Image Annotation , 2013, IEEE Transactions on Image Processing.

[30]  Daniel Rueckert,et al.  LEAP: Learning embeddings for atlas propagation , 2010, NeuroImage.

[31]  Wei Li,et al.  Automatic hippocampus segmentation of 7.0Tesla MR images by combining multiple atlases and auto-context models , 2013, NeuroImage.

[32]  Lei Wang,et al.  The Relationship of Intellectual Functioning and Cognitive Performance to Brain Structure in Schizophrenia , 2016, Schizophrenia bulletin.

[33]  J. Wegiel,et al.  775 Mathematical model of the rate of neurofibrillary changes in the hippocampal formation in end-stage Alzheimer disease , 1996, Neurobiology of Aging.

[34]  Danyang Li,et al.  Ensemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition , 2017, Cognitive Computation.

[35]  Zhidong Deng,et al.  Segmentation of Drivable Road Using Deep Fully Convolutional Residual Network with Pyramid Pooling , 2018, Cognitive Computation.

[36]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[37]  L. Squire,et al.  The medial temporal lobe and the attributes of memory , 2011, Trends in Cognitive Sciences.

[38]  Tianzi Jiang,et al.  Local label learning (LLL) for subcortical structure segmentation: Application to hippocampus segmentation , 2014, Human brain mapping.

[39]  Márcio Sarroglia Pinho,et al.  Automated Methods for Hippocampus Segmentation: the Evolution and a Review of the State of the Art , 2014, Neuroinformatics.

[40]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[41]  Johan H C Reiber,et al.  Automated Segmentation of in Vivo and Ex Vivo Mouse Brain Magnetic Resonance Images , 2009, Molecular imaging.

[42]  Azar Zandifar,et al.  A comparison of accurate automatic hippocampal segmentation methods , 2017, NeuroImage.

[43]  V. R. Steiger,et al.  Pattern of structural brain changes in social anxiety disorder after cognitive behavioral group therapy: a longitudinal multimodal MRI study , 2017, Molecular Psychiatry.

[44]  Turi O. Dalaker,et al.  Brain atrophy and disability progression in multiple sclerosis patients: a 10-year follow-up study , 2014, Journal of Neurology, Neurosurgery & Psychiatry.

[45]  Yoshua Bengio,et al.  Equilibrated adaptive learning rates for non-convex optimization , 2015, NIPS.

[46]  Rebecca C. Knickmeyer,et al.  A Structural MRI Study of Human Brain Development from Birth to 2 Years , 2008, The Journal of Neuroscience.

[47]  O. Ciccarelli,et al.  MRI CRITERIA FOR THE DIAGNOSIS OF MULTIPLE SCLEROSIS: MAGNIMS CONSENSUS GUIDELINES , 2016, The Lancet Neurology.

[48]  J. Bremner,et al.  MR-based in vivo hippocampal volumetrics: 1. Review of methodologies currently employed , 2005, Molecular Psychiatry.

[49]  Jundong Liu,et al.  Hippocampus segmentation through multi-view ensemble ConvNets , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[50]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[51]  Hamid Soltanian-Zadeh,et al.  Automatic Segmentation of Brain Structures Using Geometric Moment Invariants and Artificial Neural Networks , 2009, IPMI.

[52]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[53]  Sang Won Seo,et al.  Fully-automated approach to hippocampus segmentation using a graph-cuts algorithm combined with atlas-based segmentation and morphological opening. , 2013, Magnetic resonance imaging.

[54]  Nick C Fox,et al.  The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods , 2008, Journal of magnetic resonance imaging : JMRI.

[55]  Weifeng Liu,et al.  Multiview dimension reduction via Hessian multiset canonical correlations , 2018, Inf. Fusion.