Integrating Lung Parenchyma Segmentation and Nodule Detection With Deep Multi-Task Learning

Lung parenchyma segmentation is valuable for improving the performance of lung nodule detection in computed tomography (CT) images. Traditionally, the two tasks are performed separately. This paper proposes a deep multi-task learning (MTL) approach to integrate these tasks for better lung nodule detection. Three new ideas lead to our proposed approach. First, lung parenchyma segmentation is used as the attention module and is combined with nodule detection in a single deep network. Second, lung nodule detection is performed in an anchor-free manner by dividing it into two subtasks, nodule center identification and nodule size regression. Third, a novel pyramid dilated convolution block (PDCB) is proposed to utilize the advantage of dilated convolution and tackle its gridding problem for better lung parenchyma segmentation. Based on these ideas, we design our end-to-end deep network architecture and corresponding MTL method to achieve lung parenchyma segmentation and nodule detection simultaneously. We evaluate the proposed approach on the commonly used Lung Nodule Analysis 2016 (LUNA16) dataset. The experimental results show the value of our contributions and demonstrate that our approach can yield significant improvements compared with state-of-the-art counterparts.

[1]  Ulas Bagci,et al.  S4ND: Single-Shot Single-Scale Lung Nodule Detection , 2018, MICCAI.

[2]  Hiroshi Fujita,et al.  Fast lung nodule detection in chest CT images using cylindrical nodule-enhancement filter , 2012, International Journal of Computer Assisted Radiology and Surgery.

[3]  Zhe Li,et al.  Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Mannudeep K. Kalra,et al.  Multi-task learning for mortality prediction in LDCT images , 2020, Medical Imaging.

[5]  Youbao Tang,et al.  CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation , 2018, MICCAI.

[6]  Hao Chen,et al.  Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge , 2016, Medical Image Anal..

[7]  Klaus H. Maier-Hein,et al.  An attempt at beating the 3D U-Net , 2019, Submissions to the 2019 Kidney Tumor Segmentation Challenge: KiTS19.

[8]  João Manuel R. S. Tavares,et al.  Novel and powerful 3D adaptive crisp active contour method applied in the segmentation of CT lung images , 2017, Medical Image Anal..

[9]  Xiaohui Xie,et al.  DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Sheng Tang,et al.  Automated Pulmonary Nodule Detection: High Sensitivity with Few Candidates , 2018, MICCAI.

[11]  Brahim Ait Skourt,et al.  Lung CT Image Segmentation Using Deep Neural Networks , 2018 .

[12]  Jamshid Dehmeshki,et al.  Automated detection of lung nodules in CT images using shape-based genetic algorithm , 2007, Comput. Medical Imaging Graph..

[13]  S. Armato,et al.  Automated lung segmentation for thoracic CT impact on computer-aided diagnosis. , 2004, Academic radiology.

[14]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Geoffrey McLennan,et al.  Establishing a normative atlas of the human lung: intersubject warping and registration of volumetric CT images. , 2003, Academic radiology.

[16]  Shiliang Pu,et al.  Small-Scale Pedestrian Detection Based on Topological Line Localization and Temporal Feature Aggregation , 2018, ECCV.

[17]  Tae-Sun Choi,et al.  Automated pulmonary nodule detection based on three-dimensional shape-based feature descriptor , 2014, Comput. Methods Programs Biomed..

[18]  Hao Chen,et al.  Multi-Task Deep Model With Margin Ranking Loss for Lung Nodule Analysis , 2020, IEEE Transactions on Medical Imaging.

[19]  Jianwei Wang,et al.  Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[20]  Yuemeng Li,et al.  DeepSEED: 3D Squeeze-and-Excitation Encoder-Decoder Convolutional Neural Networks for Pulmonary Nodule Detection , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[21]  Liqiang Zhang,et al.  3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks , 2018, Canadian Conference on AI.

[22]  Yuxing Tang,et al.  XLSor: A Robust and Accurate Lung Segmentor on Chest X-Rays Using Criss-Cross Attention and Customized Radiorealistic Abnormalities Generation , 2018, MIDL.

[23]  Aurélio Campilho,et al.  Detection of juxta-pleural lung nodules in computed tomography images , 2017, Medical Imaging.

[24]  Chunliang Wang,et al.  Segmentation of Multiple Structures in Chest Radiographs Using Multi-task Fully Convolutional Networks , 2017, SCIA.

[25]  Jie Tian,et al.  Using multi-task learning to improve diagnostic performance of convolutional neural networks , 2019, Medical Imaging.

[26]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[27]  David Beymer,et al.  Automated volumetric lung segmentation of thoracic CT images using fully convolutional neural network , 2018, Medical Imaging.

[28]  B. van Ginneken,et al.  Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection. , 2009, Medical physics.

[29]  Fergus Gleeson,et al.  Efficient 3D Fully Convolutional Networks for Pulmonary Lobe Segmentation in CT Images , 2019, ArXiv.

[30]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[31]  Emanuele Pesce,et al.  Learning to detect chest radiographs containing pulmonary lesions using visual attention networks , 2017, Medical Image Anal..

[32]  Eric A. Hoffman,et al.  Automatic lung segmentation for accurate quantitation of volumetric X-ray CT images , 2001, IEEE Transactions on Medical Imaging.

[33]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[34]  Xinjian Chen,et al.  Automatic lung segmentation in low-dose CT image with contrastive attention module , 2020, Medical Imaging: Image Processing.

[35]  Qiang Zhang,et al.  Pulmonary Nodule Detection Based on Multi-branch 3D Squeeze-and-Excitation Network , 2019, 2019 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech).

[36]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[37]  Yi Yang,et al.  Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification , 2018, ArXiv.

[38]  Hei Law,et al.  CornerNet: Detecting Objects as Paired Keypoints , 2018, ECCV.

[39]  Akinobu Shimizu,et al.  Multi-shape graph cuts with neighbor prior constraints and its application to lung segmentation from a chest CT volume , 2013, Medical Image Anal..

[40]  Qiang Li,et al.  Automated segmentation of lungs with severe interstitial lung disease in CT. , 2009, Medical physics.

[41]  Rongguo Zhang,et al.  Group-Attention Single-Shot Detector (GA-SSD): Finding Pulmonary Nodules in Large-Scale CT Images , 2019, MIDL.

[42]  Aoxue Li,et al.  Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks , 2017, MICCAI.

[43]  M. Ginsberg,et al.  Epidemiology of Lung Cancer , 2013, Seminars in Interventional Radiology.

[44]  Ronald M. Summers,et al.  Progressive and Multi-path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images , 2017, MICCAI.

[45]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[46]  Sejin Park,et al.  X-ray Image Segmentation using Multi-task Learning , 2020 .

[47]  Max A. Viergever,et al.  On Combining Computer-Aided Detection Systems , 2011, IEEE Transactions on Medical Imaging.

[48]  Berkman Sahiner,et al.  3-D Convolutional Neural Networks for Automatic Detection of Pulmonary Nodules in Chest CT , 2019, IEEE Journal of Biomedical and Health Informatics.

[49]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Kim L. Boyer,et al.  Stratified learning of local anatomical context for lung nodules in CT images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Hao Chen,et al.  Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning , 2017, MICCAI.