Interpretative Computer-aided Lung Cancer Diagnosis: from Radiology Analysis to Malignancy Evaluation

BACKGROUND AND OBJECTIVE Computer-aided diagnosis (CAD) systems promote accurate diagnosis and reduce the burden of radiologists. A CAD system for lung cancer diagnosis includes nodule candidate detection and nodule malignancy evaluation. Recently, deep learning-based pulmonary nodule detection has reached satisfactory performance ready for clinical application. However, deep learning-based nodule malignancy evaluation depends on heuristic inference from low-dose computed tomography (LDCT) volume to malignant probability, and lacks clinical cognition. METHODS In this paper, we propose a joint radiology analysis and malignancy evaluation network called R2MNet to evaluate pulmonary nodule malignancy via the analysis of radiological characteristics. Radiological features are extracted as channel descriptor to highlight specific regions of the input volume that are critical for nodule malignancy evaluation. In addition, for model explanations, we propose channel-dependent activation mapping (CDAM) to visualize features and shed light on the decision process of deep neural networks (DNNs). RESULTS Experimental results on the lung image database consortium image collection (LIDC-IDRI) dataset demonstrate that the proposed method achieved an area under curve (AUC) of 96.27% and 97.52% on nodule radiology analysis and nodule malignancy evaluation, respectively. In addition, explanations of CDAM features proved that the shape and density of nodule regions are two critical factors that influence a nodule to be inferred as malignant. This process conforms to the diagnosis cognition of experienced radiologists. CONCLUSION The network inference process conforms to the diagnostic procedure of radiologists and increases the confidence of evaluation results by incorporating radiology analysis with nodule malignancy evaluation. Besides, model interpretation with CDAM features shed light on the focus regions of DNNs during the estimation of nodule malignancy probabilities.

[1]  Thomas de Quincey.  [C] , 2000, The Works of Thomas De Quincey, Vol. 1: Writings, 1799–1820.

[2]  Zijian Zhang,et al.  Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Ulas Bagci,et al.  TumorNet: Lung nodule characterization using multi-view Convolutional Neural Network with Gaussian Process , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[5]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[6]  Dennis Wollersheim,et al.  Pulmonary nodule classification with deep residual networks , 2017, International Journal of Computer Assisted Radiology and Surgery.

[7]  장윤희,et al.  Y. , 2003, Industrial and Labor Relations Terms.

[8]  Andrew A. Berlin,et al.  A 3D Probabilistic Deep Learning System for Detection and Diagnosis of Lung Cancer Using Low-Dose CT Scans , 2019, IEEE Transactions on Medical Imaging.

[9]  Guido Gerig,et al.  ITK-SNAP: An interactive tool for semi-automatic segmentation of multi-modality biomedical images , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[10]  Hao Chen,et al.  Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection , 2017, IEEE Transactions on Biomedical Engineering.

[11]  Vineeth N. Balasubramanian,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[12]  Stephen Lin,et al.  GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[13]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[14]  Xiaohui Xie,et al.  DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[15]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Taco S Cohen,et al.  Pulmonary nodule detection in CT scans with equivariant CNNs , 2019, Medical Image Anal..

[17]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  P. Alam.  ‘L’ , 2021, Composites Engineering: An A–Z Guide.

[19]  Ulas Bagci,et al.  Risk Stratification of Lung Nodules Using 3D CNN-Based Multi-task Learning , 2017, IPMI.

[20]  Maxine Tan,et al.  Lung nodule classification using deep Local–Global networks , 2019, International Journal of Computer Assisted Radiology and Surgery.

[21]  Hao Chen,et al.  Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge , 2016, Medical Image Anal..

[22]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[23]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[24]  Jianwei Wang,et al.  Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[25]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[26]  C. Mathers,et al.  Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012 , 2015, International journal of cancer.

[27]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  P. Alam.  ‘E’ , 2021, Composites Engineering: An A–Z Guide.

[29]  Miss A.O. Penney.  (b) , 1974, The New Yale Book of Quotations.

[30]  Mario Ceresa,et al.  Integration of Convolutional Neural Networks for Pulmonary Nodule Malignancy Assessment in a Lung Cancer Classification Pipeline , 2019, Comput. Methods Programs Biomed..

[31]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[32]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Wei Shen,et al.  Multi-scale Convolutional Neural Networks for Lung Nodule Classification , 2015, IPMI.

[34]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[35]  Zhe Li,et al.  Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Yu Cao,et al.  MD-NDNet: a multi-dimensional convolutional neural network for false-positive reduction in pulmonary nodule detection , 2020, Physics in medicine and biology.

[38]  Bram van Ginneken,et al.  Computer analysis of computed tomography scans of the lung: a survey , 2006, IEEE Transactions on Medical Imaging.