Pyramid Focusing Network for mutation prediction and classification in CT images

Predicting the mutation status of genes in tumors is of great clinical significance. Recent studies have suggested that certain mutations may be noninvasively predicted by studying image features of the tumors from Computed Tomography (CT) data. Currently, this kind of image feature identification method mainly relies on manual processing to extract generalized image features alone or machine processing without considering the morphological differences of the tumor itself, which makes it difficult to achieve further breakthroughs. In this paper, we propose a pyramid focusing network (PFNet) for mutation prediction and classification based on CT images. Firstly, we use Space Pyramid Pooling to collect semantic cues in feature maps from multiple scales according to the observation that the shape and size of the tumors are varied.Secondly, we improve the loss function based on the consideration that the features required for proper mutation detection are often not obvious in cross-sections of tumor edges, which raises more attention to these hard examples in the network. Finally, we devise a training scheme based on data augmentation to enhance the generalization ability of networks. Extensively verified on clinical gastric CT datasets of 20 testing volumes with 63648 CT images, our method achieves the accuracy of 94.90% in predicting the HER-2 genes mutation status of at the CT image.

[1]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Bingbing Ni,et al.  Toward automatic prediction of EGFR mutation status in pulmonary adenocarcinoma with 3D deep learning , 2019, Cancer medicine.

[3]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[4]  Lin Tian,et al.  An automatic end-to-end pipeline for CT image-based EGFR mutation status classification , 2019, Medical Imaging: Image Processing.

[5]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  J. Ross,et al.  The HER-2 receptor and breast cancer: ten years of targeted anti-HER-2 therapy and personalized medicine. , 2009, The oncologist.

[7]  Olivier Gevaert,et al.  Non–Small Cell Lung Cancer Radiogenomics Map Identifies Relationships between Molecular and Imaging Phenotypes with Prognostic Implications , 2017, Radiology.

[8]  C F Loughran,et al.  Seeding of tumour cells following breast biopsy: a literature review. , 2011, The British journal of radiology.

[9]  Y. Liu,et al.  Radiomic Features Are Associated With EGFR Mutation Status in Lung Adenocarcinomas. , 2016, Clinical lung cancer.

[10]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[11]  Ghassan Hamarneh,et al.  Noninvasive Determination of Gene Mutations in Clear Cell Renal Cell Carcinoma Using Multiple Instance Decisions Aggregated CNN , 2018, MICCAI.

[12]  William Pao,et al.  Nomogram to predict the presence of EGFR activating mutation in lung adenocarcinoma , 2011, European Respiratory Journal.

[13]  John Quackenbush,et al.  Somatic Mutations Drive Distinct Imaging Phenotypes in Lung Cancer. , 2017, Cancer research.

[14]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  H. Hricak,et al.  Radiogenomics of clear cell renal cell carcinoma: associations between CT imaging features and mutations. , 2013, Radiology.

[17]  Hassan Maleki,et al.  PET/CT Radiomic Sequencer for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients , 2018, 2018 IEEE Nuclear Science Symposium and Medical Imaging Conference Proceedings (NSS/MIC).

[18]  Paul Kinahan,et al.  Radiomics: Images Are More than Pictures, They Are Data , 2015, Radiology.

[19]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[20]  Olivier Gevaert,et al.  Non-small cell lung cancer: identifying prognostic imaging biomarkers by leveraging public gene expression microarray data--methods and preliminary results. , 2012, Radiology.

[21]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[22]  D. Dong,et al.  Predicting EGFR mutation status in lung adenocarcinoma on computed tomography image using deep learning , 2019, European Respiratory Journal.

[23]  D. Dong,et al.  Quantitative Biomarkers for Prediction of Epidermal Growth Factor Receptor Mutation in Non-Small Cell Lung Cancer , 2017, Translational oncology.