DC-Al GAN: Pseudoprogression and True Tumor Progression of Glioblastoma multiform Image Classification Based On DCGAN and Alexnet

PURPOSE Pseudoprogression (PsP) occurs in 20-30% of patients with glioblastoma multiforme (GBM) after receiving the standard treatment. PsP exhibits similarities in shape and intensity to the true tumor progression (TTP) of GBM on the follow-up magnetic resonance imaging (MRI). These similarities pose challenges to the differentiation of these types of progression and hence the selection of the appropriate clinical treatment strategy. METHODS To address this challenge, we introduced a novel feature learning method based on deep convolutional generative adversarial network (DCGAN) and AlexNet, termed DC-AL GAN, to discriminate between PsP and TTP in MRI images. Due to the adversarial relationship between the generator and the discriminator of DCGAN, high-level discriminative features of PsP and TTP can be derived for the discriminator with AlexNet. We also constructed a multi-feature selection module to concatenate features from different layers, contributing to more powerful features used for effectively discriminating between PsP and TTP. Finally, these discriminative features from the discriminator are used for classification by a support vector machine (SVM). Tenfold cross-validation (CV) and the area under the receiver operating characteristic (AUC) were applied to evaluate the performance of this developed algorithm. RESULTS The accuracy and AUC of DC-AL GAN for discriminating PsP and TTP after tenfold CV were 0.920 and 0.947. We also assessed the effects of different indicators (such as sensitivity and specificity) for features extracted from different layers to obtain a model with the best classification performance. CONCLUSIONS The proposed model DC-AL GAN is capable of learning discriminative representations from GBM datasets, and it achieves desirable PsP and TTP classification performance superior to other state-of-the-art methods. Therefore, the developed model would be useful in the diagnosis of PsP and TTP for GBM.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  Lin Zhu,et al.  Generative Adversarial Networks for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[4]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[5]  Honglak Lee,et al.  A unified framework for automatic wound segmentation and analysis with deep convolutional neural networks , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[6]  Wen Wang,et al.  Combination of IVIM-DWI and 3D-ASL for differentiating true progression from pseudoprogression of Glioblastoma multiforme after concurrent chemoradiotherapy: study protocol of a prospective diagnostic trial , 2017, BMC Medical Imaging.

[7]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[8]  Dieta Brandsma,et al.  Incidence of early pseudo‐progression in a cohort of malignant glioma patients treated with chemoirradiation with temozolomide , 2008, Cancer.

[9]  Ho Sung Kim,et al.  Different diagnostic values of imaging parameters to predict pseudoprogression in glioblastoma subgroups stratified by MGMT promoter methylation , 2016, European Radiology.

[10]  A G Sorensen,et al.  Pseudoprogression and Pseudoresponse: Imaging Challenges in the Assessment of Posttreatment Glioma , 2011, American Journal of Neuroradiology.

[11]  John T. Guibas,et al.  Synthetic Medical Images from Dual Generative Adversarial Networks , 2017, ArXiv.

[12]  S. Leenstra,et al.  Recent advances in the molecular understanding of glioblastoma , 2009 .

[13]  Kenneth R. Hess,et al.  Characterization of pseudoprogression in patients with glioblastoma: is histology the gold standard? , 2010, Journal of Neuro-Oncology.

[14]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[15]  Gelareh Zadeh,et al.  The Diagnosis and Treatment of Pseudoprogression, Radiation Necrosis and Brain Tumor Recurrence , 2014, International journal of molecular sciences.

[16]  Tracy T Batchelor,et al.  Low incidence of pseudoprogression by imaging in newly diagnosed glioblastoma patients treated with cediranib in combination with chemoradiation. , 2013, The oncologist.

[17]  P. Wen,et al.  Effect of adding temozolomide to radiation therapy on the incidence of pseudo-progression , 2009, Journal of Neuro-Oncology.

[18]  Xiaogang Wang,et al.  Medical image classification with convolutional neural network , 2014, 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV).

[19]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[20]  Dan Hu,et al.  Semisupervised Hyperspectral Image Classification Based on Generative Adversarial Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[21]  Jost Tobias Springenberg,et al.  Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks , 2015, ICLR.

[22]  M. Mehta,et al.  Pseudoprogression after glioma therapy: a comprehensive review , 2013, Expert review of neurotherapeutics.

[23]  Jonathan Li,et al.  Generative Adversarial Networks and Probabilistic Graph Models for Hyperspectral Image Classification , 2018, AAAI.

[24]  Rajan Jain,et al.  Morphologic MRI features, diffusion tensor imaging and radiation dosimetric analysis to differentiate pseudo-progression from early tumor progression , 2013, Journal of Neuro-Oncology.

[25]  S. Kim,et al.  Differentiation of Tumor Progression from Pseudoprogression in Patients with Posttreatment Glioblastoma Using Multiparametric Histogram Analysis , 2014, American Journal of Neuroradiology.

[26]  Jian Zhang,et al.  Stratification of pseudoprogression and true progression of glioblastoma multiform based on longitudinal diffusion tensor imaging without segmentation , 2016, Medical physics.

[27]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Michael Blum,et al.  High-Throughput Classification of Radiographs Using Deep Convolutional Neural Networks , 2016, Journal of Digital Imaging.

[30]  David Westhead,et al.  Prediction of clinical outcome in glioblastoma using a biologically relevant nine‐microRNA signature , 2015, Molecular oncology.

[31]  D. Kong,et al.  Diagnostic Dilemma of Pseudoprogression in the Treatment of Newly Diagnosed Glioblastomas: The Role of Assessing Relative Cerebral Blood Flow Volume and Oxygen-6-Methylguanine-DNA Methyltransferase Promoter Methylation Status , 2011, American Journal of Neuroradiology.

[32]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[33]  Augustus Odena,et al.  Semi-Supervised Learning with Generative Adversarial Networks , 2016, ArXiv.

[34]  Ghassem Tofighi,et al.  Classification of Alzheimer's Disease using fMRI Data and Deep Learning Convolutional Neural Networks , 2016, ArXiv.

[35]  Sanjeev Chawla,et al.  Three‐dimensional echo planar spectroscopic imaging for differentiation of true progression from pseudoprogression in patients with glioblastoma , 2018, NMR in biomedicine.

[36]  John R. Smith,et al.  Deep Learning, Sparse Coding, and SVM for Melanoma Recognition in Dermoscopy Images , 2015, MLMI.

[37]  Hideo Negishi,et al.  The IRF family of transcription factors , 2012, Oncoimmunology.

[38]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[39]  J D Tucker,et al.  Characterization of the XRCC1-DNA ligase III complex in vitro and its absence from mutant hamster cells. , 1995, Nucleic acids research.

[40]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[41]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[42]  A. Brandes,et al.  MGMT promoter methylation status can predict the incidence and outcome of pseudoprogression after concomitant radiochemotherapy in newly diagnosed glioblastoma patients. , 2008, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[43]  Jianfeng Xu,et al.  Genomic predictors of patterns of progression in glioblastoma and possible influences on radiation field design , 2015, Journal of Neuro-Oncology.

[44]  Tom White,et al.  Generative Adversarial Networks: An Overview , 2017, IEEE Signal Processing Magazine.

[45]  Tae Min Kim,et al.  Differentiation of true progression from pseudoprogression in glioblastoma treated with radiation therapy and concomitant temozolomide: comparison study of standard and high-b-value diffusion-weighted imaging. , 2013, Radiology.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Stefano Ermon,et al.  Semi-Supervised Multitask Learning on Multispectral Satellite Images Using Wasserstein Generative Adversarial Networks (GANs) for Predicting Poverty , 2019, ArXiv.

[48]  C. Suh,et al.  Prediction of Pseudoprogression in Patients with Glioblastomas Using the Initial and Final Area Under the Curves Ratio Derived from Dynamic Contrast-Enhanced T1-Weighted Perfusion MR Imaging , 2013, American Journal of Neuroradiology.

[49]  Klaus Schöffmann,et al.  Learning laparoscopic video shot classification for gynecological surgery , 2018, Multimedia Tools and Applications.

[50]  Shu Liao,et al.  Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition , 2016, IEEE Transactions on Medical Imaging.

[51]  Tae Min Kim,et al.  Usefulness of MS-MLPA for detection of MGMT promoter methylation in the evaluation of pseudoprogression in glioblastoma patients. , 2011, Neuro-oncology.

[52]  O. De Witte,et al.  High levels of cellular proliferation predict pseudoprogression in glioblastoma patients , 2011, International journal of oncology.

[53]  Namkug Kim,et al.  Which combination of MR imaging modalities is best for predicting recurrent glioblastoma? Study of diagnostic accuracy and reproducibility. , 2014, Radiology.

[54]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Jianwei Zhao,et al.  Automatic detection and classification of leukocytes using convolutional neural networks , 2016, Medical & Biological Engineering & Computing.

[56]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[57]  Kiyohiro Houkin,et al.  IDH1 mutation as a potential novel biomarker for distinguishing pseudoprogression from true progression in patients with glioblastoma treated with temozolomide and radiotherapy , 2013, Brain Tumor Pathology.

[58]  Liangpei Zhang,et al.  Pre-Trained AlexNet Architecture with Pyramid Pooling and Supervision for High Spatial Resolution Remote Sensing Image Scene Classification , 2017, Remote. Sens..

[59]  Max A. Viergever,et al.  Automatic segmentation of the left ventricle in cardiac CT angiography using convolutional neural networks , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[60]  S. Brem,et al.  Differentiating Tumor Progression from Pseudoprogression in Patients with Glioblastomas Using Diffusion Tensor Imaging and Dynamic Susceptibility Contrast MRI , 2015, American Journal of Neuroradiology.

[61]  M. Markey,et al.  Differentiating tumor recurrence from treatment necrosis: a review of neuro-oncologic imaging strategies. , 2013, Neuro-oncology.

[62]  Hyun-Cheol Kang,et al.  Pseudoprogression in patients with malignant gliomas treated with concurrent temozolomide and radiotherapy: potential role of p53 , 2011, Journal of Neuro-Oncology.