Topological structure and global features enhanced graph reasoning model for non-small cell lung cancer segmentation from CT

Objective. Accurate and automated segmentation of lung tumors from computed tomography (CT) images is critical yet challenging. Lung tumors are of various sizes and locations and have indistinct boundaries adjacent to other normal tissues. Approach. We propose a new segmentation model that can integrate the topological structure and global features of image region nodes to address the challenges. Firstly, we construct a weighted graph with image region nodes. The graph topology reflects the complex spatial relationships among these nodes, and each node has its specific attributes. Secondly, we propose a node-wise topological feature learning module based on a new graph convolutional autoencoder (GCA). Meanwhile, a node information supplementation (GNIS) module is established by integrating specific features of each node extracted by a convolutional neural network (CNN) into each encoding layer of GCA. Afterwards, we construct a global feature extraction model based on multi-layer perceptron (MLP) to encode the features learnt from all the image region nodes which are crucial complementary information for tumor segmentation. Main results. Ablation study results over the public lung tumor segmentation dataset demonstrate the contributions of our major technical innovations. Compared with other segmentation methods, our new model improves the segmentation performance and has generalization ability on different 3D image segmentation backbones. Our model achieved Dice of 0.7827, IoU of 0.6981, and HD of 32.1743 mm on the public dataset 2018 Medical Segmentation Decathlon challenge, and Dice of 0.7004, IoU of 0.5704 and HD of 64.4661 mm on lung tumor dataset from Shandong Cancer Hospital. Significance. The novel model improves automated lung tumor segmentation performance especially the challenging and complex cases using topological structure and global features of image region nodes. It is of great potential to apply the model to other CT segmentation tasks.

[1]  L. Geng,et al.  Review of Deep Learning Based Automatic Segmentation for Lung Cancer Radiotherapy , 2021, Frontiers in Oncology.

[2]  A. Jemal,et al.  Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries , 2021, CA: a cancer journal for clinicians.

[3]  Changming Sun,et al.  Cascade knowledge diffusion network for skin lesion diagnosis and segmentation , 2021, Appl. Soft Comput..

[4]  S. Manoharan,et al.  Early diagnosis of Lung Cancer with Probability of Malignancy Calculation and Automatic Segmentation of Lung CT scan Images , 2020, December 2020.

[5]  Jens Petersen,et al.  nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation , 2020, Nature Methods.

[6]  Toshiya Nakaguchi,et al.  Graph Convolutional Autoencoder and Fully-Connected Autoencoder with Attention Mechanism Based Method for Predicting Drug-Disease Associations , 2020, IEEE Journal of Biomedical and Health Informatics.

[7]  Yalin Zheng,et al.  CNN-GCN Aggregation Enabled Boundary Regression for Biomedical Image Segmentation , 2020, MICCAI.

[8]  Fang Wang,et al.  Multimodal Priors Guided Segmentation of Liver Lesions in MRI Using Mutual Information Based Graph Co-Attention Networks , 2020, MICCAI.

[9]  Hong Liu,et al.  Spatial Pyramid Based Graph Reasoning for Semantic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Xiao Wang,et al.  Structural Deep Clustering Network , 2020, WWW.

[11]  Simon Fong,et al.  Automated 3-D lung tumor detection and classification by an active contour model and CNN classifier , 2019, Expert Syst. Appl..

[12]  Taesup Kim,et al.  Scalable Neural Architecture Search for 3D Medical Image Segmentation , 2019, MICCAI.

[13]  Boqiang Liu,et al.  HSN: Hybrid Segmentation Network for Small Cell Lung Cancer Segmentation , 2019, IEEE Access.

[14]  Shuicheng Yan,et al.  Graph-Based Global Reasoning Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Changming Sun,et al.  RA-UNet: A Hybrid Deep Attention-Aware Network to Extract Liver and Tumor in CT Scans , 2018, Frontiers in Bioengineering and Biotechnology.

[16]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[17]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[18]  Daniël M Pelt,et al.  A mixed-scale dense convolutional neural network for image analysis , 2017, Proceedings of the National Academy of Sciences.

[19]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[22]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[23]  Trevor Darrell,et al.  Fully convolutional networks for semantic segmentation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[25]  Wei Chen,et al.  MAU-Net: Multiple Attention 3D U-Net for Lung Cancer Segmentation on CT Images , 2021, KES.