Higher-order Transfer Learning for Pulmonary Nodule Attribute Prediction in Chest CT Images

Attributes like texture, lobulation, malignancy, etc., are commonly used to describe the phenotype of a pulmonary nodule in computed tomography (CT) image, which can provide useful medical knowledge for the identification of early stage lung cancer. There may exist certain relations among these attributes, and some attributes may naturally imply or boost others that have been less comprehensively exploited in previous studies. In this paper, we explicitly model the relations among 11 attributes of nodules by way of transfer learning and extract a meta-structure that captures the transferabilities across deep features of these attributes. Specifically, a higher-order transfer learning scheme is proposed by involving three phases, i.e., semantic attribute-specific modeling, semantic attributes transfer modeling and pathologic attribute generalizing, to explore the strongest association across various attributes and to boost the nodule attribute predictions in chest CT images. The proposed approach has been evaluated on the 2632 nodules in the public Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset. The experimental results suggest that our higher-order transfer approach shows the superior predictive performance not only in the most of the semantic attributes compared with the schemes of learning from scratch and the first-order transfer but also for the pathologic attribute compared with the related studies. In addition, we demonstrate an attribute transfer graph to reveal which attributes combination can supply the most useful information to boost the predictive performance of target attributes.

[1]  Alexander Wong,et al.  Lung Nodule Classification Using Deep Features in CT Images , 2015, 2015 12th Conference on Computer and Robot Vision.

[2]  Wei Shen,et al.  Learning from Experts: Developing Transferable Deep Features for Patient-Level Lung Cancer Prediction , 2016, MICCAI.

[3]  Bo Chen,et al.  Low-Shot Multi-label Incremental Learning for Thoracic Diseases Diagnosis , 2018, ICONIP.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Masoom A. Haider,et al.  Discovery Radiomics for Pathologically-Proven Computed Tomography Lung Cancer Prediction , 2015, ICIAR.

[6]  Leonidas J. Guibas,et al.  Taskonomy: Disentangling Task Transfer Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Bai Ying Lei,et al.  Automatic Scoring of Multiple Semantic Attributes With Multi-Task Feature Leverage: A Study on Pulmonary Nodules in CT Images , 2017, IEEE Transactions on Medical Imaging.

[8]  Wei Shen,et al.  Lung nodule malignancy prediction using multi-task convolutional neural network , 2017, Medical Imaging.

[9]  Ronald M. Summers,et al.  Deep Lesion Graphs in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-Scale Lesion Database , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[11]  Ying Zhou,et al.  WGAN-Based Synthetic Minority Over-Sampling Technique: Improving Semantic Fine-Grained Classification for Lung Nodules in CT Images , 2019, IEEE Access.

[12]  Jun Huang,et al.  Multi-order Transfer Learning for Pathologic Diagnosis of Pulmonary Nodule Malignancy , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[13]  Mei Xie,et al.  Automatic Categorization and Scoring of Solid, Part-Solid and Non-Solid Pulmonary Nodules in CT Images with Convolutional Neural Network , 2017, Scientific Reports.

[14]  Gang Wang,et al.  Multi-Task CNN Model for Attribute Prediction , 2015, IEEE Transactions on Multimedia.

[15]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.