Deep Semi Supervised Generative Learning for Automated Tumor Proportion Scoring on NSCLC Tissue Needle Biopsies

The level of PD-L1 expression in immunohistochemistry (IHC) assays is a key biomarker for the identification of Non-Small-Cell-Lung-Cancer (NSCLC) patients that may respond to anti PD-1/PD-L1 treatments. The quantification of PD-L1 expression currently includes the visual estimation by a pathologist of the percentage (tumor proportional scoring or TPS) of tumor cells showing PD-L1 staining. Known challenges like differences in positivity estimation around clinically relevant cut-offs and sub-optimal quality of samples makes visual scoring tedious and subjective, yielding a scoring variability between pathologists. In this work, we propose a novel deep learning solution that enables the first automated and objective scoring of PD-L1 expression in late stage NSCLC needle biopsies. To account for the low amount of tissue available in biopsy images and to restrict the amount of manual annotations necessary for training, we explore the use of semi-supervised approaches against standard fully supervised methods. We consolidate the manual annotations used for training as well the visual TPS scores used for quantitative evaluation with multiple pathologists. Concordance measures computed on a set of slides unseen during training provide evidence that our automatic scoring method matches visual scoring on the considered dataset while ensuring repeatability and objectivity.

[1]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[2]  Nicole Schechter,et al.  Development of a programmed cell death ligand-1 immunohistochemical assay validated for analysis of non-small cell lung cancer and head and neck squamous cell carcinoma , 2016, Diagnostic Pathology.

[3]  Alan Sharpe,et al.  Agreement between Programmed Cell Death Ligand-1 Diagnostic Assays across Multiple Protein Expression Cutoffs in Non–Small Cell Lung Cancer , 2017, Clinical Cancer Research.

[4]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[5]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[6]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Thomas J. Fuchs,et al.  Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in Pathology , 2018, ArXiv.

[8]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[9]  Eric Faulkner,et al.  PD-L1 diagnostic tests: a systematic literature review of scoring algorithms and test-validation metrics , 2018, Diagnostic Pathology.

[10]  Hamid R. Tizhoosh,et al.  Convolutional neural networks for histopathology image classification: Training vs. Using pre-trained networks , 2017, 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA).

[11]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[13]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[14]  Hyunseok Min,et al.  Neural Stain-Style Transfer Learning using GAN for Histopathological Images , 2017, ArXiv.

[15]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[16]  Keith M. Kerr,et al.  PL 03.03 Blueprint 2: PD-L1 Immunohistochemistry Comparability Study in Real-Life, Clinical Samples , 2017 .

[17]  Raphaël Marée,et al.  Comparison of Deep Transfer Learning Strategies for Digital Pathology , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  S. Fox,et al.  Comparison of Four PD‐L1 Immunohistochemical Assays in Lung Cancer , 2017, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[20]  F. Hirsch,et al.  PD‐L1 Immunohistochemistry Assays for Lung Cancer: Results from Phase 1 of the Blueprint PD‐L1 IHC Assay Comparison Project , 2017, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[23]  Michel E. Vandenberghe,et al.  Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer , 2017, Scientific Reports.

[24]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Daisuke Komura,et al.  Machine Learning Methods for Histopathological Image Analysis , 2017, Computational and structural biotechnology journal.

[26]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[27]  Jedd D. Wolchok,et al.  PD-L1 (B7-H1) and PD-1 pathway blockade for cancer therapy: Mechanisms, response biomarkers, and combinations , 2016, Science Translational Medicine.

[28]  Matteo Brunelli,et al.  PD-L1 Assays 22C3 and SP263 are Not Interchangeable in Non–Small Cell Lung Cancer When Considering Clinically Relevant Cutoffs: An Interclone Evaluation by Differently Trained Pathologists , 2018, The American journal of surgical pathology.

[29]  Nicholas J. Vogelzang,et al.  Efficacy and Safety of Durvalumab in Locally Advanced or Metastatic Urothelial Carcinoma: Updated Results From a Phase 1/2 Open-label Study , 2017, JAMA oncology.

[30]  Hui Yu,et al.  PD‐L1 Immunohistochemistry Comparability Study in Real‐Life Clinical Samples: Results of Blueprint Phase 2 Project , 2018, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[31]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[32]  N. Rizvi,et al.  PD-L1 biomarker testing for non-small cell lung cancer: truth or fiction? , 2016, Journal of Immunotherapy for Cancer.

[33]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[36]  S. Russel and P. Norvig,et al.  “Artificial Intelligence – A Modern Approach”, Second Edition, Pearson Education, 2003. , 2015 .

[37]  Kibok Lee,et al.  Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification , 2016, ICML.

[38]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[39]  Yuanjie Zheng,et al.  Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model , 2017, Scientific Reports.

[40]  M. Dolled-Filhart,et al.  Development of a Companion Diagnostic PD-L1 Immunohistochemistry Assay for Pembrolizumab Therapy in Non–Small-cell Lung Cancer , 2016, Applied immunohistochemistry & molecular morphology : AIMM.

[41]  Hyojin Kim,et al.  PD-L1 immunohistochemical assays for assessment of therapeutic strategies involving immune checkpoint inhibitors in non-small cell lung cancer: a comparative study , 2017, Oncotarget.

[42]  L. Tanoue,et al.  Lung cancer: epidemiology, etiology, and prevention. , 2011, Clinics in chest medicine.

[43]  C. Rudin,et al.  Nivolumab versus Docetaxel in Advanced Nonsquamous Non-Small-Cell Lung Cancer. , 2015, The New England journal of medicine.

[44]  Anant Madabhushi,et al.  Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent , 2017, Scientific Reports.

[45]  B. van Ginneken,et al.  Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis , 2016, Scientific Reports.

[46]  C. Shu,et al.  PD-L1 as a biomarker in NSCLC: challenges and future directions. , 2017, Annals of translational medicine.

[47]  Anant Madabhushi,et al.  Out-of-Sample Extrapolation utilizing Semi-Supervised Manifold Learning (OSE-SSL): Content Based Image Retrieval for Histopathology Images , 2016, Scientific Reports.

[48]  Dorit Merhof,et al.  Analyzing Immunohistochemically Stained Whole-Slide Images of Ovarian Carcinoma , 2017, Bildverarbeitung für die Medizin.

[49]  Anne L. Martel,et al.  A Cluster-then-label Semi-supervised Learning Approach for Pathology Image Classification , 2018, Scientific Reports.

[50]  Nassir Navab,et al.  Staingan: Stain Style Transfer for Digital Histological Images , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).