Semi-supervised adversarial model for benign-malignant lung nodule classification on chest CT

Classification of benign-malignant lung nodules on chest CT is the most critical step in the early detection of lung cancer and prolongation of patient survival. Despite their success in image classification, deep convolutional neural networks (DCNNs) always require a large number of labeled training data, which are not available for most medical image analysis applications due to the work required in image acquisition and particularly image annotation. In this paper, we propose a semi-supervised adversarial classification (SSAC) model that can be trained by using both labeled and unlabeled data for benign-malignant lung nodule classification. This model consists of an adversarial autoencoder-based unsupervised reconstruction network R, a supervised classification network C, and learnable transition layers that enable the adaption of the image representation ability learned by R to C. The SSAC model has been extended to the multi-view knowledge-based collaborative learning, aiming to employ three SSACs to characterize each nodule's overall appearance, heterogeneity in shape and texture, respectively, and to perform such characterization on nine planar views. The MK-SSAC model has been evaluated on the benchmark LIDC-IDRI dataset and achieves an accuracy of 92.53% and an AUC of 95.81%, which are superior to the performance of other lung nodule classification and semi-supervised learning approaches.

[1]  Raunak Dey,et al.  Diagnostic classification of lung nodules using 3D neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[2]  Daniel Cremers,et al.  Learning by Association — A Versatile Semi-Supervised Training Method for Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  A. Jemal,et al.  Lung Cancer Statistics. , 2016, Advances in experimental medicine and biology.

[4]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[5]  Nassir Navab,et al.  Semi-supervised Deep Learning for Fully Convolutional Networks , 2017, MICCAI.

[6]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[7]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[8]  Yanning Zhang,et al.  Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT , 2018, Inf. Fusion.

[9]  David Dagan Feng,et al.  Classification of Medical Images in the Biomedical Literature by Jointly Using Deep and Handcrafted Visual Features , 2018, IEEE Journal of Biomedical and Health Informatics.

[10]  Wen-Huang Cheng,et al.  Computer-aided classification of lung nodules on computed tomography images via deep learning technique , 2015, OncoTargets and therapy.

[11]  Weidong Cai,et al.  Knowledge-based Collaborative Deep Learning for Benign-Malignant Lung Nodule Classification on Chest CT , 2019, IEEE Transactions on Medical Imaging.

[12]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[13]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[14]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[15]  Raúl Rojas,et al.  The Backpropagation Algorithm , 1996 .

[16]  David Dagan Feng,et al.  Atlas registration and ensemble deep convolutional neural network-based prostate segmentation using magnetic resonance imaging , 2018, Neurocomputing.

[17]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[18]  Ernst J. Rummeny,et al.  Multiparametric MR and PET Imaging of Intratumoral Biological Heterogeneity in Patients with Metastatic Lung Cancer Using Voxel-by-Voxel Analysis , 2015, PloS one.

[19]  J. Alison Noble,et al.  &OHgr;‐Net (Omega‐Net): Fully automatic, multi‐view cardiac MR detection, orientation, and segmentation with deep neural networks☆ , 2018, Medical Image Anal..

[20]  David Dagan Feng,et al.  Transferable Multi-model Ensemble for Benign-Malignant Lung Nodule Classification on Chest CT , 2017, MICCAI.

[21]  Xiaolin Huang,et al.  Pulmonary nodule segmentation with CT sample synthesis using adversarial networks , 2019, Medical physics.

[22]  D. Raz,et al.  Lung Cancer Screening. , 2016, Cancer treatment and research.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Marc'Aurelio Ranzato,et al.  Semi-supervised learning of compact document representations with deep networks , 2008, ICML '08.

[25]  Karen Drukker,et al.  LUNGx Challenge for computerized lung nodule classification: reflections and lessons learned. , 2015, Journal of medical imaging.

[26]  D. Berry,et al.  Benefits and harms of CT screening for lung cancer: a systematic review. , 2012, JAMA.

[27]  Ulas Bagci,et al.  Risk Stratification of Lung Nodules Using 3D CNN-Based Multi-task Learning , 2017, IPMI.

[28]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[29]  Niranjan Khandelwal,et al.  A Combination of Shape and Texture Features for Classification of Pulmonary Nodules in Lung CT Images , 2016, Journal of Digital Imaging.

[30]  Renda Soylemez Wiener,et al.  What is a Lung Nodule? , 2016, American journal of respiratory and critical care medicine.

[31]  Xin Geng,et al.  Classification of Lung Nodule Malignancy Risk on Computed Tomography Images Using Convolutional Neural Network: A Comparison Between 2D and 3D Strategies , 2016, ACCV Workshops.

[32]  Kun Zhao,et al.  Lung nodule classification by the combination of fusion classifier and cascaded convolutional neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[33]  Hao Chen,et al.  3D multi‐scale FCN with random modality voxel dropout learning for Intervertebral Disc Localization and Segmentation from Multi‐modality MR Images , 2018, Medical Image Anal..

[34]  Sidong Liu,et al.  Lung Nodule Classification by Jointly Using Visual Descriptors and Deep Features , 2016, MCV/BAMBI@MICCAI.

[35]  Hao Chen,et al.  Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection , 2017, IEEE Transactions on Biomedical Engineering.

[36]  Xin Zhao,et al.  Semi-Supervised Multimodal Deep Learning for RGB-D Object Recognition , 2016, IJCAI.

[37]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[38]  O. Chapelle,et al.  Semi-Supervised Learning (Chapelle, O. et al., Eds.; 2006) [Book reviews] , 2009, IEEE Transactions on Neural Networks.

[39]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[40]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[41]  Tapani Raiko,et al.  Semi-supervised Learning with Ladder Networks , 2015, NIPS.

[42]  Lin Yang,et al.  Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images , 2017, MICCAI.

[43]  Robert Hecht-Nielsen III.3 – Theory of the Backpropagation Neural Network* , 1992 .

[44]  Anant Madabhushi,et al.  Intra-perinodular Textural Transition (Ipris): A 3D Descriptor for Nodule Diagnosis on Lung CT , 2017, MICCAI.

[45]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[46]  Wei Shen,et al.  Multi-crop Convolutional Neural Networks for lung nodule malignancy suspiciousness classification , 2017, Pattern Recognit..

[47]  Ulas Bagci,et al.  Characterization of Lung Nodule Malignancy Using Hybrid Shape and Appearance Features , 2016, MICCAI.

[48]  Hong Zhao,et al.  Texture Feature Analysis for Computer-Aided Diagnosis on Pulmonary Nodules , 2015, Journal of Digital Imaging.

[49]  Hao Chen,et al.  Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning , 2017, MICCAI.

[50]  Noel C. F. Codella,et al.  Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC) , 2016, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[51]  Zhenyu Liu,et al.  Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation , 2017, Medical Image Anal..

[52]  Yong Xia,et al.  Attention Residual Learning for Skin Lesion Classification , 2019, IEEE Transactions on Medical Imaging.

[53]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[54]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[55]  Wei Shen,et al.  Multi-scale Convolutional Neural Networks for Lung Nodule Classification , 2015, IPMI.

[56]  Ulas Bagci,et al.  TumorNet: Lung nodule characterization using multi-view Convolutional Neural Network with Gaussian Process , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).