Hybrid Attention Densely Connected Ensemble Framework for Lesion Segmentation From Magnetic Resonance Images

White matter hyperintensities (WMHs) are associated with various neurological and aging diseases, and morphological analysis plays a crucial role in the assessment of disease progression. In this article, a novel hybrid attention densely connected ensemble framework is deployed for WMH segmentation from multi-modality magnetic resonance imaging (MRI). On the one hand, hybrid attention densely convolutional network (HA-DCN) is designed with a novel hybrid attention module embedded. The hybrid attention module can further improve the precision of lesion localization by extracting the complementary information of high-level features and low-level features from the spatial domain and channel domain. On the other hand, the focal Tversky loss function and generalized dice loss function derived from dice similarity coefficient are ensembled into the proposed framework, which achieves a trade-off between specificity and sensitivity. As a result, the volume of automation lesion segmentation is more agreeable to the manual lesion segmentation by experts. The proposed framework was evaluated online and offline in the MICCAI 2017 WMH segmentation challenge. A quantitative experiment has further demonstrated the effect of multi-modality and the effectiveness of the proposed hybrid attention densely connected ensemble framework. Furthermore, the challenge dataset consists of three scanners, reflecting the flexibility and robustness of the model. It also exhibits its potential for real-world clinical practice.

[1]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[2]  Daniel Cremers,et al.  FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture , 2016, ACCV.

[3]  Karl J. Friston,et al.  Unified segmentation , 2005, NeuroImage.

[4]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[5]  Pengfei Xiong,et al.  Pyramid Attention Network for Semantic Segmentation , 2018, BMVC.

[6]  Tat-Seng Chua,et al.  Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks , 2017, IJCAI.

[7]  Koen L. Vincken,et al.  Probabilistic segmentation of white matter lesions in MR imaging , 2004, NeuroImage.

[8]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[9]  Hang Li,et al.  Deep learning for natural language processing: advantages and challenges , 2018 .

[10]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[11]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[12]  Elena Marchiori,et al.  Location Sensitive Deep Convolutional Neural Networks for Segmentation of White Matter Hyperintensities , 2016, Scientific Reports.

[13]  Guixia Kang,et al.  Multi-target Interactive Neural Network for Automated Segmentation of the Hippocampus in Magnetic Resonance Imaging , 2019, Cognitive Computation.

[14]  Tao Song,et al.  Integrated Extractor, Generator and Segmentor for Ischemic Stroke Lesion Segmentation , 2018, BrainLes@MICCAI.

[15]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Rohan Ramanath,et al.  An Attentive Survey of Attention Models , 2019, ACM Trans. Intell. Syst. Technol..

[17]  Daniel L. Rubin,et al.  Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions , 2017, Journal of Digital Imaging.

[18]  Yunchao Wei,et al.  CCNet: Criss-Cross Attention for Semantic Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[20]  J. Pluim,et al.  Evaluation of a deep learning approach for the segmentation of brain tissues and white matter hyperintensities of presumed vascular origin in MRI , 2017, NeuroImage: Clinical.

[21]  Klaus H. Maier-Hein,et al.  nnU-Net: Breaking the Spell on Successful Medical Image Segmentation , 2019, ArXiv.

[22]  G. Barker,et al.  Quantification of MRI lesion load in multiple sclerosis: a comparison of three computer-assisted techniques. , 1996, Magnetic resonance imaging.

[23]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[24]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[25]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  P. Scheltens,et al.  White matter hyperintensities, cognitive impairment and dementia: an update , 2015, Nature Reviews Neurology.

[27]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[28]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Wei-Shi Zheng,et al.  Fully convolutional network ensembles for white matter hyperintensities segmentation in MR images , 2018, NeuroImage.

[30]  Mengyu Liu,et al.  Feature Pyramid Encoding Network for Real-time Semantic Segmentation , 2019, BMVC.

[31]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Nicu Sebe,et al.  Multi-Channel Attention Selection GAN With Cascaded Semantic Guidance for Cross-View Image Translation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Alex Rovira,et al.  Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach , 2017, NeuroImage.

[34]  Cristian Vicas,et al.  Deep convolutional neural nets for objective steatosis detection from liver samples , 2017, 2017 13th IEEE International Conference on Intelligent Computer Communication and Processing (ICCP).

[35]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[36]  Kevin Murphy,et al.  Attention-Based Extraction of Structured Information from Street View Imagery , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[37]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[38]  Simon K. Warfield,et al.  A Model of Population and Subject (MOPS) Intensities With Application to Multiple Sclerosis Lesion Segmentation , 2015, IEEE Transactions on Medical Imaging.

[39]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[40]  Boris C. Bernhardt,et al.  Deep Convolutional Networks for Automated Detection of Epileptogenic Brain Malformations , 2018, MICCAI.

[41]  Olivier Clatz,et al.  Spatial Decision Forests for MS Lesion Segmentation in Multi-Channel MR Images , 2010, MICCAI.

[42]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[43]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[44]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Hao Chen,et al.  VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images , 2017, NeuroImage.

[46]  Naimul Mefraz Khan,et al.  A Novel Focal Tversky Loss Function With Improved Attention U-Net for Lesion Segmentation , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[47]  Nick C Fox,et al.  Neuroimaging standards for research into small vessel disease and its contribution to ageing and neurodegeneration , 2013, The Lancet Neurology.

[48]  Olivier Clatz,et al.  Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images , 2011, NeuroImage.

[49]  Antonios Danelakis,et al.  Survey of automated multiple sclerosis lesion segmentation techniques on magnetic resonance imaging , 2018, Comput. Medical Imaging Graph..

[50]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[51]  Simon Andermatt,et al.  Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge , 2019, IEEE Transactions on Medical Imaging.