Shape and margin-aware lung nodule classification in low-dose CT images via soft activation mapping

A number of studies on lung nodule classification lack clinical/biological interpretations of the features extracted by convolutional neural network (CNN). The methods like class activation mapping (CAM) and gradient-based CAM (Grad-CAM) are tailored for interpreting localization and classification tasks while they ignored fine-grained features. Therefore, CAM and Grad-CAM cannot provide optimal interpretation for lung nodule categorization task in low-dose CT images, in that fine-grained pathological clues like discrete and irregular shape and margins of nodules are capable of enhancing sensitivity and specificity of nodule classification with regards to CNN. In this paper, we first develop a soft activation mapping (SAM) to enable fine-grained lung nodule shape & margin (LNSM) feature analysis with a CNN so that it can access rich discrete features. Secondly, by combining high-level convolutional features with SAM, we further propose a high-level feature enhancement scheme (HESAM) to localize LNSM features. Experiments on the LIDC-IDRI dataset indicate that 1) SAM captures more fine-grained and discrete attention regions than existing methods, 2) HESAM localizes more accurately on LNSM features and obtains the state-of-the-art predictive performance, reducing the false positive rate, and 3) we design and conduct a visually matching experiment which incorporates radiologists study to increase the confidence level of applying our method to clinical diagnosis.

[1]  Yutaka Satoh,et al.  Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet? , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Jonas Kubilius,et al.  Deep Neural Networks as a Computational Model for Human Shape Sensitivity , 2016, PLoS Comput. Biol..

[3]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[4]  Wei Shen,et al.  Multi-scale Convolutional Neural Networks for Lung Nodule Classification , 2015, IPMI.

[5]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[6]  Ahmed Hosny,et al.  Artificial intelligence in radiology , 2018, Nature Reviews Cancer.

[7]  Taco S Cohen,et al.  Pulmonary nodule detection in CT scans with equivariant CNNs , 2019, Medical Image Anal..

[8]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[9]  J. Mechalakos,et al.  Radiomics analysis of pulmonary nodules in low‐dose CT for early detection of lung cancer , 2018, Medical physics.

[10]  Uwe Kruger,et al.  Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose CT image reconstruction , 2019, Nat. Mach. Intell..

[11]  Samuel Ritter,et al.  Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study , 2017, ICML.

[12]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[13]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[14]  Patrick Granton,et al.  Radiomics: extracting more information from medical images using advanced feature analysis. , 2012, European journal of cancer.

[15]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[16]  Rebecca S Lewis,et al.  Projected cancer risks from computed tomographic scans performed in the United States in 2007. , 2009, Archives of internal medicine.

[17]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Qi Wu,et al.  Medical image classification using synergic deep learning , 2019, Medical Image Anal..

[19]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[20]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[21]  C. Gatsonis,et al.  Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening , 2012 .

[22]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[23]  Hu Chen,et al.  Low-dose CT via convolutional neural network. , 2017, Biomedical optics express.

[24]  Yiming Lei,et al.  A novel approach for cirrhosis recognition via improved LBP algorithm and dictionary learning , 2017, Biomed. Signal Process. Control..

[25]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[26]  D. Brenner,et al.  Computed tomography--an increasing source of radiation exposure. , 2007, The New England journal of medicine.

[27]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[28]  Wojciech Czarnecki,et al.  On Loss Functions for Deep Neural Networks in Classification , 2017, ArXiv.

[29]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[30]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  K. Doi,et al.  Malignant versus benign nodules at CT screening for lung cancer: comparison of thin-section CT findings. , 2004, Radiology.

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Jianwei Wang,et al.  Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[36]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[37]  Linda B. Smith,et al.  The importance of shape in early lexical learning , 1988 .

[38]  Kin Keung Lai,et al.  A Bias-Variance-Complexity Trade-Off Framework for Complex System Modeling , 2006, ICCSA.

[39]  Shuicheng Yan,et al.  Dual Path Networks , 2017, NIPS.

[40]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[41]  Hongming Shan,et al.  3-D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning From a 2-D Trained Network , 2018, IEEE Transactions on Medical Imaging.

[42]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[43]  Ben Glocker,et al.  Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images , 2018, Medical Image Anal..

[44]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[45]  Ulas Bagci,et al.  TumorNet: Lung nodule characterization using multi-view Convolutional Neural Network with Gaussian Process , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[46]  Xiaohui Xie,et al.  DeepLung: 3D Deep Convolutional Nets for Automated Pulmonary Nodule Detection and Classification , 2017, bioRxiv.

[47]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[48]  Emanuele Pesce,et al.  Learning to detect chest radiographs containing pulmonary lesions using visual attention networks , 2017, Medical Image Anal..

[49]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).