Fast and fully-automated detection and segmentation of pulmonary nodules in thoracic CT scans using deep convolutional neural networks

Deep learning techniques have been extensively used in computerized pulmonary nodule analysis in recent years. Many reported studies still utilized hybrid methods for diagnosis, in which convolutional neural networks (CNNs) are used only as one part of the pipeline, and the whole system still needs either traditional image processing modules or human intervention to obtain final results. In this paper, we introduced a fast and fully-automated end-to-end system that can efficiently segment precise lung nodule contours from raw thoracic CT scans. Our proposed system has four major modules: candidate nodule detection with Faster regional-CNN (R-CNN), candidate merging, false positive (FP) reduction with CNN, and nodule segmentation with customized fully convolutional neural network (FCN). The entire system has no human interaction or database specific design. The average runtime is about 16 s per scan on a standard workstation. The nodule detection accuracy is 91.4% and 94.6% with an average of 1 and 4 false positives (FPs) per scan. The average dice coefficient of nodule segmentation compared to the groundtruth is 0.793.

[1]  Xiaohui Xie,et al.  DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[2]  Fangfang Dong,et al.  Brain MR image segmentation based on local Gaussian mixture model and nonlocal spatial regularization , 2014, J. Vis. Commun. Image Represent..

[3]  Wenqing Sun,et al.  Automatic lung nodule graph cuts segmentation with deep learning false positive reduction , 2017, Medical Imaging.

[4]  Naixue Xiong,et al.  Steganalysis of LSB matching using differences between nonadjacent pixels , 2016, Multimedia Tools and Applications.

[5]  Antoni B. Chan,et al.  On measuring the change in size of pulmonary nodules , 2006, IEEE Transactions on Medical Imaging.

[6]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[7]  Russell C. Hardie,et al.  Automatic segmentation of small pulmonary nodules in computed tomography data using a radial basis function neural network with application to volume estimation , 2008 .

[8]  Yifei Zhang,et al.  A novel approach of lung segmentation on chest CT images using graph cuts , 2015, Neurocomputing.

[9]  Bram van Ginneken,et al.  Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images , 2014, Medical Image Anal..

[10]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[11]  W. Qian,et al.  Mini-array of multiple tumor-associated antigens (TAAs) in the immunodiagnosis of breast cancer , 2012, Oncology letters.

[12]  Shoji Kido,et al.  Automatic segmentation of pulmonary nodules on CT images by use of NCI lung image database consortium , 2006, SPIE Medical Imaging.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Wenqing Sun,et al.  Balance the nodule shape and surroundings: a new multichannel image based convolutional neural network scheme on lung nodule diagnosis , 2017, Medical Imaging.

[15]  Hao Chen,et al.  Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection , 2017, IEEE Transactions on Biomedical Engineering.

[16]  A. Jemal,et al.  Cancer statistics, 2018 , 2018, CA: a cancer journal for clinicians.

[17]  Alan D. Lopez,et al.  Global, Regional, and National Cancer Incidence, Mortality, Years of Life Lost, Years Lived With Disability, and Disability-Adjusted Life-years for 32 Cancer Groups, 1990 to 2015: A Systematic Analysis for the Global Burden of Disease Study , 2017, JAMA oncology.

[18]  K. Doi,et al.  Computerized detection of lung nodules in thin-section CT images by use of selective enhancement filters and an automated rule-based classifier. , 2008, Academic Radiology.

[19]  Ian Goodfellow,et al.  Generative adversarial networks , 2020, Commun. ACM.

[20]  Rebecca L. Siegel Mph,et al.  Cancer statistics, 2018 , 2018 .

[21]  Marcelo Gattass,et al.  Automatic segmentation of lung nodules with growing neural gas and support vector machine , 2012, Comput. Biol. Medicine.

[22]  L. Schwartz,et al.  Segmentation of lung lesions on CT scans using watershed, active contours, and Markov random field. , 2013, Medical physics.

[23]  Lubomir M. Hadjiiski,et al.  Computer-aided detection of lung nodules: false positive reduction using a 3D gradient field method and 3D ellipsoid fitting. , 2005, Medical physics.

[24]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Lanfen Lin,et al.  A deep 3D residual CNN for false-positive reduction in pulmonary nodule detection. , 2018, Medical physics.

[26]  Bram van Ginneken,et al.  Supervised Probabilistic Segmentation of Pulmonary Nodules in CT Scans , 2006, MICCAI.

[27]  Giovanni Montana,et al.  Recurrent Convolutional Networks for Pulmonary Nodule Detection in CT Imaging , 2016, ArXiv.

[28]  Hao Chen,et al.  Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning , 2017, MICCAI.

[29]  M. Botvinick,et al.  Neural representations of events arise from temporal community structure , 2013, Nature Neuroscience.

[30]  Zhenyu Liu,et al.  Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation , 2017, Medical Image Anal..

[31]  Wenqing Sun,et al.  Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis , 2017, Comput. Biol. Medicine.

[32]  Menglong Yan,et al.  Semantic Segmentation of Aerial Images With Shuffling Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[33]  Bram van Ginneken,et al.  Automatic detection of large pulmonary solid nodules in thoracic CT images. , 2015, Medical physics.

[34]  Lu Yang,et al.  Semantic Segmentation for High Spatial Resolution Remote Sensing Images Based on Convolution Neural Network and Pyramid Pooling Module , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Nima Tajbakhsh,et al.  Comparing two classes of end-to-end machine-learning models in lung nodule detection and classification: MTANNs vs. CNNs , 2017, Pattern Recognit..

[36]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[37]  Ying Zhang,et al.  Boundary delineation in transrectal ultrasound image for prostate cancer , 2007, Comput. Biol. Medicine.

[38]  Berkman Sahiner,et al.  3D convolutional neural network for automatic detection of lung nodules in chest CT , 2017, Medical Imaging.

[39]  Wenqing Sun.  Deep Learning Method Vs. Hand-Crafted Features For Lung Cancer Diagnosis And Breast Cancer Risk Analysis , 2017 .

[40]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[41]  Hyojin Kim,et al.  Lung nodule detection using 3D convolutional neural networks trained on weakly labeled data , 2016, SPIE Medical Imaging.

[42]  Taghi M. Khoshgoftaar,et al.  A survey of transfer learning , 2016, Journal of Big Data.

[43]  Wei Qian,et al.  Effect of wavelet bases on compressing digital mammograms , 1995 .

[44]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[45]  Zdenka Babić,et al.  Using GANs to Enable Semantic Segmentation of Ranging Sensor Data , 2018, 2018 Zooming Innovation in Consumer Technologies Conference (ZINC).

[46]  Wei Qian,et al.  Order statistic-neural network hybrid filters for gamma camera-bremsstrahlung image restoration , 1993, IEEE Trans. Medical Imaging.

[47]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[48]  Temesguen Messay,et al.  Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the Lung Image Database Consortium and Image Database Resource Initiative dataset , 2015, Medical Image Anal..

[49]  Aoxue Li,et al.  Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks , 2017, MICCAI.

[50]  Jamshid Dehmeshki,et al.  Segmentation of Pulmonary Nodules in Thoracic CT Scans: A Region Growing Approach , 2008, IEEE Transactions on Medical Imaging.

[51]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[52]  C. Gatsonis,et al.  Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening , 2012 .

[53]  Bram van Ginneken,et al.  A large-scale evaluation of automatic pulmonary nodule detection in chest CT using local image features and k-nearest-neighbour classification , 2009, Medical Image Anal..

[54]  Jan Cornelis,et al.  A novel computer-aided lung nodule detection system for CT images. , 2011, Medical physics.

[55]  Taco Cohen,et al.  3D G-CNNs for Pulmonary Nodule Detection , 2018, ArXiv.

[56]  Vivek Vaidya,et al.  Lung nodule detection in CT using 3D convolutional neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[57]  T. Sørensen,et al.  A method of establishing group of equal amplitude in plant sociobiology based on similarity of species content and its application to analyses of the vegetation on Danish commons , 1948 .

[58]  D. Warren,et al.  TESTING ECOLOGICAL EXPLANATIONS FOR BIOGEOGRAPHIC BOUNDARIES , 2011, Evolution; international journal of organic evolution.

[59]  Wenqing Sun,et al.  Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data , 2017, Comput. Medical Imaging Graph..

[60]  Tao Gong,et al.  An Effective Hybrid Windowed Fourier Filtering and Fuzzy C-Mean for Pulmonary Nodule Segmentation , 2018 .

[61]  Nicolas Guizard,et al.  CASED: Curriculum Adaptive Sampling for Extreme Data Imbalance , 2017, MICCAI.

[62]  V. Vijaya Kishore,et al.  Performance evaluation of edge detectors - morphology based ROI segmentation and nodule detection from DICOM lung images in the noisy environment , 2013, 2013 3rd IEEE International Advance Computing Conference (IACC).

[63]  Jianwei Wang,et al.  Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[64]  Marcos Salganicoff,et al.  Segmentation of pulmonary nodules of various densities with morphological approaches and convexity models , 2011, Medical Image Anal..

[65]  Caiming Zhang,et al.  A fast weak-supervised pulmonary nodule segmentation method based on modified self-adaptive FCM algorithm , 2017, Soft Computing.

[66]  Jun Zhao,et al.  Automatic detection of lung nodules: false positive reduction using convolution neural networks and handcrafted features , 2017, Medical Imaging.

[67]  Wenqing Sun,et al.  Computer aided lung cancer diagnosis with deep learning algorithms , 2016, SPIE Medical Imaging.

[68]  Qian Wang,et al.  Segmentation of lung nodules in computed tomography images using dynamic programming and multidirection fusion techniques. , 2009, Academic radiology.

[69]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[70]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[71]  Juan de Lara,et al.  Supporting user-oriented analysis for multi-view domain-specific visual languages , 2009, Inf. Softw. Technol..

[72]  Dansheng Song,et al.  Ipsilateral-mammogram computer-aided detection of breast cancer. , 2004, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[73]  B. van Ginneken,et al.  Robust semi-automatic segmentation of pulmonary subsolid nodules in chest computed tomography scans , 2015, Physics in medicine and biology.

[74]  Max A. Viergever,et al.  On Combining Computer-Aided Detection Systems , 2011, IEEE Transactions on Medical Imaging.

[75]  Eyal Oren,et al.  Global, Regional, and National Cancer Incidence, Mortality, Years of Life Lost, Years Lived With Disability, and Disability-Adjusted Life-years for 32 Cancer Groups, 1990 to 2015 , 2016 .