A semi-supervised convolutional transfer neural network for 3D pulmonary nodules detection

Abstract Data-driven segmentation has been a challenging task in automatic pulmonary nodules detection. Most methods for data-driven segmentation require training examples labeled with segmentation masks. This requirement makes it expensive to annotate new categories, especially in most cases, since only weak label information can be obtained from CT. This paper proposes a new semi-supervised 3D deep neural network to solve this problem. The model outputs all suspicious nodules for a subject by training this network using image-level tag annotations from the DSB dataset and mask annotations from the LUNA dataset. We evaluate our approach in a controlled study on the DSB test dataset. The over-fitting caused by the shortage of training data is alleviated by training two modules alternatively. This model can synchronously extract tag labels and mask labels in a sequence, and combine two objects through one weighted transfer function in the network learning process. The experimental results demonstrate the power of using the proposed method to produce significant improvements in the accuracy of pulmonary nodules detection.

[1]  Hao Chen,et al.  Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection , 2017, IEEE Transactions on Biomedical Engineering.

[2]  Trevor Darrell,et al.  Constrained Convolutional Neural Networks for Weakly Supervised Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Ilaria Gori,et al.  Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: The ANODE09 study , 2010, Medical Image Anal..

[4]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[5]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Peter V. Gehler,et al.  A Generative Model of People in Clothing , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Hao Chen,et al.  Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge , 2016, Medical Image Anal..

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jia Xu,et al.  Learning to segment under various forms of weak supervision , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[12]  Geoffrey McLennan,et al.  Assessment of radiologist performance in the detection of lung nodules: dependence on the definition of "truth". , 2009, Academic radiology.

[13]  Lin Yang,et al.  Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation , 2016, NIPS.

[14]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Cordelia Schmid,et al.  Multi-region Two-Stream R-CNN for Action Detection , 2016, ECCV.

[16]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Zhe Li,et al.  Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Vladlen Koltun,et al.  Photographic Image Synthesis with Cascaded Refinement Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[20]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[21]  Mingzhe Liu,et al.  A new local density and relative distance based spectrum clustering , 2019, Knowledge and Information Systems.

[22]  Björn Ommer,et al.  A Variational U-Net for Conditional Appearance and Shape Generation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Jian Sun,et al.  BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Xun Wang,et al.  Review on mining data from multiple data sources , 2018, Pattern Recognit. Lett..

[25]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Trevor Darrell,et al.  Learning to Segment Every Thing , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Aoxue Li,et al.  Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks , 2017, MICCAI.

[28]  Armando Santoro,et al.  A randomized study of lung cancer screening with spiral computed tomography: three-year results from the DANTE trial. , 2009, American journal of respiratory and critical care medicine.

[29]  Yi Ding,et al.  RUN: Residual U-Net for Computer-Aided Detection of Pulmonary Nodules without Candidate Selection , 2018, ArXiv.

[30]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).