Brain Tumor Segmentation Network Using Attention-based Fusion and Spatial Relationship Constraint

Delineating the brain tumor from magnetic resonance (MR) images is critical for the treatment of gliomas. However, automatic delineation is challenging due to the complex appearance and ambiguous outlines of tumors. Considering that multi-modal MR images can reflect different tumor biological properties, we develop a novel multi-modal tumor segmentation network (MMTSN) to robustly segment brain tumors based on multi-modal MR images. The MMTSN is composed of three sub-branches and a main branch. Specifically, the sub-branches are used to capture different tumor features from multi-modal images, while in the main branch, we design a spatial-channel fusion block (SCFB) to effectively aggregate multi-modal features. Additionally, inspired by the fact that the spatial relationship between sub-regions of tumor is relatively fixed, e.g., the enhancing tumor is always in the tumor core, we propose a spatial loss to constrain the relationship between different sub-regions of tumor. We evaluate our method on the test set of multi-modal brain tumor segmentation challenge 2020 (BraTs2020). The method achieves 0.8764, 0.8243 and 0.773 dice score for whole tumor, tumor core and enhancing tumor, respectively.

[1]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[2]  Na Li,et al.  Automated brain tumor segmentation from multi-modality MRI data based on Tamura texture feature and SVM model , 2019, Journal of Physics: Conference Series.

[3]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[4]  Guoyan Zheng,et al.  3 D multi-scale FCN with random modality voxel dropout learning for Intervertebral Disc Localization and Segmentation from Multimodality MR Images , 2018 .

[5]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[6]  Yong Yin,et al.  MMFNet: A Multi-modality MRI Fusion Network for Segmentation of Nasopharyngeal Carcinoma , 2018, Neurocomputing.

[7]  Su Ruan,et al.  A review: Deep learning for medical image segmentation using multi-modality fusion , 2019, Array.

[8]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[9]  Sim Heng Ong,et al.  Focus, Segment and Erase: An Efficient Network for Multi-label Brain Tumor Segmentation , 2018, ECCV.

[10]  Sébastien Ourselin,et al.  Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks , 2017, BrainLes@MICCAI.

[11]  Dacheng Tao,et al.  One-Pass Multi-task Convolutional Neural Networks for Efficient Brain Tumor Segmentation , 2018, MICCAI.

[12]  Hao Chen,et al.  3D multi‐scale FCN with random modality voxel dropout learning for Intervertebral Disc Localization and Segmentation from Multi‐modality MR Images , 2018, Medical Image Anal..

[13]  Xiahai Zhuang,et al.  Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[15]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[16]  Santosh Kesari,et al.  Malignant gliomas in adults. , 2008, The New England journal of medicine.

[17]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[18]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.