Pre-trained deep convolutional neural networks for the segmentation of malignant pleural mesothelioma tumor on CT scans

Pre-trained deep convolutional neural networks (CNNs) have shown promise in the training of deep CNNs for medical imaging applications. The purpose of this study was to investigate the use of partially pre-trained deep CNNs for the segmentation of malignant pleural mesothelioma tumor on CT scans. Four network configurations were investigated: (1) VGG16/U-Net network with pre-trained layers fixed during training, (2) VGG16/U-Net network with pre-trained layers fine-tuned during training, (3) VGG16/U-Net network with all except the first two pre-trained layers fine-tuned during training, and (4) a standard U-Net architecture trained from scratch. Deep CNNs were trained separately for tumor segmentation in left and right hemithoraces using 4259 and 6441 contoured axial CT sections, respectively. A test set of 61 CT sections from 16 patients excluded from training was used to evaluate segmentation performance; the Dice similarity coefficient (DSC) was calculated between computer-generated and reference segmentations provided by two radiologists and one radiology resident. Median DSC on the test set was 0.739 (range 0.328–0.920), 0.772 (range 0.342–0.949), 0.777 (range 0.216–0.946), and 0.758 (range 0.099–0.943) across all observers for network configurations (1), (2), (3) and (4) above, respectively. The median DSC achieved with configuration (3) when compared with the standard U-Net trained from scratch was found to be significantly higher for two out of three observers. A fine-tuned VGG16/U-Net deep CNN showed significantly higher overlap with two out of three observers when compared with a standard U-Net trained from scratch for the segmentation of malignant pleural mesothelioma tumor.

[1]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[2]  David C Rice,et al.  A Multicenter Study of Volumetric Computed Tomography for Staging Malignant Pleural Mesothelioma. , 2016, The Annals of thoracic surgery.

[3]  Samuel G. Armato,et al.  Revised Modified Response Evaluation Criteria in Solid Tumors for Assessment of Response in Malignant Pleural Mesothelioma (Version 1.1) , 2018, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[4]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[5]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  H. Kindler,et al.  Update on malignant pleural mesothelioma. , 2011, Seminars in respiratory and critical care medicine.

[8]  H. Kindler,et al.  Malignant pleural mesothelioma: an update on biomarkers and treatment. , 2009, Chest.

[9]  B. Seifert,et al.  Volumetry: an alternative to assess therapy response for malignant pleural mesothelioma? , 2011, European Respiratory Journal.

[10]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[11]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[12]  Binsheng Zhao,et al.  Assessment of Therapy Responses and Prediction of Survival in Malignant Pleural Mesothelioma Through Computer-Aided Volumetric Measurement on Computed Tomography Scans , 2010, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[13]  Alexey Shvets,et al.  TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation , 2018, Computer-Aided Analysis of Gastrointestinal Videos.

[14]  Samuel G Armato,et al.  Computerized segmentation and measurement of malignant pleural mesothelioma. , 2010, Medical physics.

[15]  Natalia Antropova,et al.  A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets , 2017, Medical physics.

[16]  S. Steinberg,et al.  Preoperative tumor volume is associated with outcome in malignant pleural mesothelioma. , 1998, The Journal of thoracic and cardiovascular surgery.

[17]  Samuel G Armato,et al.  Modeling of mesothelioma growth demonstrates weaknesses of current response criteria. , 2006, Lung cancer.

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[20]  S. Armato,et al.  Disease volumes as a marker for patient response in malignant pleural mesothelioma. , 2013, Annals of oncology : official journal of the European Society for Medical Oncology.

[21]  Claude Denham,et al.  Phase III study of pemetrexed in combination with cisplatin versus cisplatin alone in patients with malignant pleural mesothelioma. , 2003, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  M. Byrne,et al.  Modified RECIST criteria for assessment of response in malignant pleural mesothelioma. , 2004, Annals of oncology : official journal of the European Society for Medical Oncology.

[24]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[25]  Samuel G Armato,et al.  Deep convolutional neural networks for the automated segmentation of malignant pleural mesothelioma on computed tomography scans , 2018, Journal of medical imaging.

[26]  Shin Matsuoka,et al.  Epithelial malignant pleural mesothelioma after extrapleural pneumonectomy: stratification of survival with CT-derived tumor volume. , 2012, AJR. American journal of roentgenology.

[27]  Samuel G Armato,et al.  Lung Volume Measurements as a Surrogate Marker for Patient Response in Malignant Pleural Mesothelioma , 2013, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[28]  Hariharan Ravishankar,et al.  Understanding the Mechanisms of Deep Transfer Learning for Medical Images , 2016, LABELS/DLMIA@MICCAI.

[29]  Luca Maria Gambardella,et al.  High-Performance Neural Networks for Visual Object Classification , 2011, ArXiv.

[30]  John P Lichtenberger,et al.  Multimodality imaging for characterization, classification, and staging of malignant pleural mesothelioma. , 2014, Radiographics : a review publication of the Radiological Society of North America, Inc.