E2Net: An Edge Enhanced Network for Accurate Liver and Tumor Segmentation on CT Scans

Developing an effective liver and liver tumor segmentation model from CT scans is very important for the success of liver cancer diagnosis, surgical planning and cancer treatment. In this work, we propose a two-stage framework for 2D liver and tumor segmentation. The first stage is a coarse liver segmentation network, while the second stage is an edge enhanced network (E$^2$Net) for more accurate liver and tumor segmentation. E$^2$Net explicitly models complementary objects (liver and tumor) and their edge information within the network to preserve the organ and lesion boundaries. We introduce an edge prediction module in E$^2$Net and design an edge distance map between liver and tumor boundaries, which is used as an extra supervision signal to train the edge enhanced network. We also propose a deep cross feature fusion module to refine multi-scale features from both objects and their edges. E$^2$Net is more easily and efficiently trained with a small labeled dataset, and it can be trained/tested on the original 2D CT slices (resolve resampling error issue in 3D models). The proposed framework has shown superior performance on both liver and liver tumor segmentation compared to several state-of-the-art 2D, 3D and 2D/3D hybrid frameworks.

[1]  Hans Meine,et al.  Neural Network-Based Automatic Liver Tumor Segmentation With Random Forest-Based Candidate Filtering , 2017, ArXiv.

[2]  Hao Chen,et al.  Light-Weight Hybrid Convolutional Network for Liver Tumor Segmentation , 2019, IJCAI.

[3]  Syamsiah Mashohor,et al.  Automatic liver segmentation on Computed Tomography using random walkers for treatment planning , 2016, EXCLI journal.

[4]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[5]  Youbao Tang,et al.  CT Image Enhancement Using Stacked Generative Adversarial Networks and Transfer Learning for Lesion Segmentation Improvement , 2018, MLMI@MICCAI.

[6]  Hans Meine,et al.  Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing , 2018, Scientific Reports.

[7]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[8]  Zhiyong Lu,et al.  Automated abnormality classification of chest radiographs using deep convolutional neural networks , 2020, npj Digital Medicine.

[9]  Kai Zhao,et al.  Res2Net: A New Multi-Scale Backbone Architecture , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Youbao Tang,et al.  MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentation , 2019, MICCAI.

[11]  Yen-Wei Chen,et al.  Improved segmentation of low-contrast lesions using sigmoid edge model , 2016, International Journal of Computer Assisted Radiology and Surgery.

[12]  Youbao Tang,et al.  One Click Lesion RECIST Measurement and Segmentation on CT Scans , 2020, MICCAI.

[13]  Youbao Tang,et al.  Weakly Supervised Lesion Co-Segmentation on Ct Scans , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[14]  Gregory D. Hager,et al.  Anatomy-Aware Siamese Network: Exploiting Semantic Asymmetry for Accurate Pelvic Fracture Detection in X-ray Images , 2020, ECCV.

[15]  Nuno Vasconcelos,et al.  Volumetric Attention for 3D Medical Image Segmentation and Detection , 2019, MICCAI.

[16]  S. Jagannath,et al.  Tumor burden assessment and its implication for a prognostic model in advanced diffuse large-cell lymphoma. , 1986, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[17]  Yuxing Tang,et al.  TUNA-Net: Task-oriented UNsupervised Adversarial Network for Disease Recognition in Cross-Domain Chest X-rays , 2019, MICCAI.

[18]  Youbao Tang,et al.  Semi-Automatic RECIST Labeling on CT Scans with Cascaded Convolutional Neural Networks , 2018, MICCAI.

[19]  Xiao Han,et al.  Automatic Liver Lesion Segmentation Using A Deep Convolutional Neural Network Method , 2017, ArXiv.

[20]  Youbao Tang,et al.  Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST , 2018, MICCAI.

[21]  Yuxing Tang,et al.  CT-realistic data augmentation using generative adversarial network for robust lymph node segmentation , 2019, Medical Imaging.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Yuxing Tang,et al.  Uldor: A Universal Lesion Detector For Ct Scans With Pseudo Masks And Hard Negative Example Mining , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[24]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[25]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[27]  Samuel Kadoury,et al.  Liver lesion segmentation informed by joint liver segmentation , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[28]  R. Summers,et al.  Abnormal Chest X-Ray Identification With Generative Adversarial One-Class Classifier , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[29]  Adam P. Harrison,et al.  Weakly Supervised Universal Fracture Detection in Pelvic X-rays , 2019, MICCAI.

[30]  Yuxing Tang,et al.  Cross-Domain Medical Image Translation by Shared Latent Gaussian Mixture Model , 2020, MICCAI.

[31]  Hao Chen,et al.  The Liver Tumor Segmentation Benchmark (LiTS) , 2019, Medical Image Anal..

[32]  Yuxing Tang,et al.  XLSor: A Robust and Accurate Lung Segmentor on Chest X-Rays Using Criss-Cross Attention and Customized Radiorealistic Abnormalities Generation , 2018, MIDL.

[33]  Youbao Tang,et al.  Weakly-supervised lesion segmentation on CT scans using co-segmentation , 2020, Medical Imaging.

[34]  Yanhua Zhang,et al.  3D Liver Tumor Segmentation in CT Images Using Improved Fuzzy C-Means and Graph Cuts , 2017, BioMed research international.

[35]  Youbao Tang,et al.  CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation , 2018, MICCAI.

[36]  Niels Frimodt-Møller,et al.  Chromosome replication as a measure of bacterial growth rate during Escherichia coli infection in the mouse peritonitis model , 2018, Scientific Reports.

[37]  Yang Wang,et al.  Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation , 2016, ISVC.