ConvPath: A Software Tool for Lung Adenocarcinoma Digital Pathological Image Analysis Aided by Convolutional Neural Network

Background The spatial distributions of different types of cells could reveal a cancer cell's growth pattern, its relationships with the tumor microenvironment and the immune response of the body, all of which represent key “hallmarks of cancer”. However, the process by which pathologists manually recognize and localize all the cells in pathology slides is extremely labor intensive and error prone. Methods In this study, we developed an automated cell type classification pipeline, ConvPath, which includes nuclei segmentation, convolutional neural network-based tumor cell, stromal cell, and lymphocyte classification, and extraction of tumor microenvironment-related features for lung cancer pathology images. To facilitate users in leveraging this pipeline for their research, all source scripts for ConvPath software are available at https://qbrc.swmed.edu/projects/cnn/. Findings The overall classification accuracy was 92.9% and 90.1% in training and independent testing datasets, respectively. By identifying cells and classifying cell types, this pipeline can convert a pathology image into a “spatial map” of tumor, stromal and lymphocyte cells. From this spatial map, we can extract features that characterize the tumor micro-environment. Based on these features, we developed an image feature-based prognostic model and validated the model in two independent cohorts. The predicted risk group serves as an independent prognostic factor, after adjusting for clinical variables that include age, gender, smoking status, and stage. Interpretation The analysis pipeline developed in this study could convert the pathology image into a “spatial map” of tumor cells, stromal cells and lymphocytes. This could greatly facilitate and empower comprehensive analysis of the spatial organization of cells, as well as their roles in tumor progression and metastasis.

[1]  W. Weidner,et al.  Fraktale Geometrie zur Objektivierung des Gradings beim Prostatakarzinom , 2014, Der Urologe.

[2]  G. D. de Bock,et al.  The prognostic influence of tumour-infiltrating lymphocytes in cancer: a systematic review with meta-analysis , 2011, British Journal of Cancer.

[3]  Jesse K. McKenney,et al.  Validation of whole slide imaging for frozen section diagnosis in surgical pathology , 2015, Journal of pathology informatics.

[4]  Jeffrey W. Pollard,et al.  Macrophage Diversity Enhances Tumor Progression and Metastasis , 2010, Cell.

[5]  Gwénaël Le Teuff,et al.  Prognostic Effect of Tumor Lymphocytic Infiltration in Resectable Non-Small-Cell Lung Cancer. , 2016, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[6]  Zoe Wainer,et al.  Does Lung Adenocarcinoma Subtype Predict Patient Survival?: A Clinicopathologic Study Based on the New International Association for the Study of Lung Cancer/American Thoracic Society/European Respiratory Society International Multidisciplinary Lung Adenocarcinoma Classification , 2011, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[7]  Z. Werb,et al.  Tumors as organs: complex tissues that interface with the entire organism. , 2010, Developmental cell.

[8]  N. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Transactions on Medical Imaging.

[9]  Lei Guo,et al.  Microvessel prediction in H&E Stained Pathology Images using fully convolutional neural networks , 2018, BMC Bioinformatics.

[10]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[11]  David R Jones,et al.  Solid Predominant Histologic Subtype in Resected Stage I Lung Adenocarcinoma Is an Independent Predictor of Early, Extrathoracic, Multisite Recurrence and of Poor Postrecurrence Survival. , 2015, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[12]  J. Austin,et al.  The 2015 World Health Organization Classification of Lung Tumors: Impact of Genetic, Clinical and Radiologic Advances Since the 2004 Classification. , 2015, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[13]  A. Huisman,et al.  Automatic Nuclei Segmentation in H&E Stained Breast Cancer Histopathology Images , 2013, PloS one.

[14]  Anindya Sarkar,et al.  Prostate Cancer Grading: Use of Graph Cut and Spatial Arrangement of Nuclei , 2014, IEEE Transactions on Medical Imaging.

[15]  Lei Zhang,et al.  Active contours with selective local or global segmentation: A new formulation and level set method , 2010, Image Vis. Comput..

[16]  Siddharth Joshi,et al.  Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines , 2015, Front. Neurosci..

[17]  Peter Bankhead,et al.  QuPath: Open source software for digital pathology image analysis , 2017, Scientific Reports.

[18]  Kristian Pietras,et al.  Hallmarks of cancer: interactions with the tumor stroma. , 2010, Experimental cell research.

[19]  Masahiro Tsuboi,et al.  The ratio of cancer cells to stroma within the invasive area is a histologic prognostic parameter of lung adenocarcinoma. , 2018, Lung cancer.

[20]  A. Ruifrok,et al.  Quantification of histochemical staining by color deconvolution. , 2001, Analytical and quantitative cytology and histology.

[21]  Rajarsi R. Gupta,et al.  Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. , 2018, Cell reports.

[22]  Abdul Jalil,et al.  Novel structural descriptors for automated colon cancer detection and grading , 2015, Comput. Methods Programs Biomed..

[23]  Christos Hatzis,et al.  Quantitative assessment of the spatial heterogeneity of tumor-infiltrating lymphocytes in breast cancer , 2016, Breast Cancer Research.

[24]  Jakob Nikolas Kather,et al.  Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer , 2019, Nature Medicine.

[25]  Masahiro Yamaguchi,et al.  Computational grading of hepatocellular carcinoma using multifractal feature description , 2013, Comput. Medical Imaging Graph..

[26]  Faming Liang,et al.  A BAYESIAN MARK INTERACTION MODEL FOR ANALYSIS OF TUMOR PATHOLOGY IMAGES. , 2018, The annals of applied statistics.

[27]  Zhigang Yao,et al.  Deep Learning Model as a New Trend in Computer-aided Diagnosis of Tumor Pathology for Lung Cancer , 2020, Journal of Cancer.

[28]  Fabio Grizzi,et al.  Spatial distribution of B cells predicts prognosis in human pancreatic adenocarcinoma , 2016, Oncoimmunology.

[29]  G. Giamas,et al.  Targeting tumor–stroma crosstalk: the example of the NT157 inhibitor , 2016, Oncogene.

[30]  Faliu Yi,et al.  A Bayesian hidden Potts mixture model for analyzing lung cancer pathology images. , 2019, Biostatistics.

[31]  Shidan Wang,et al.  Pathology Image Analysis Using Segmentation Deep Learning Algorithms. , 2019, The American journal of pathology.

[32]  I. Sohn,et al.  Imaging Phenotyping Using Radiomics to Predict Micropapillary Pattern within Lung Adenocarcinoma , 2017, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[33]  T. Blankenstein,et al.  Immunotherapy: target the stroma to hit the tumor. , 2005, Trends in molecular medicine.

[34]  Holger Gerhardt,et al.  Pericytes limit tumor cell metastasis. , 2006, The Journal of clinical investigation.

[35]  Thomas Brox,et al.  Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[37]  Koichi Yamazaki,et al.  CD4+ T cells in cancer stroma, not CD8+ T cells in cancer cell nests, are associated with favorable prognosis in human non‐small cell lung cancers , 2003, Cancer science.

[38]  Darren Treanor,et al.  Digital pathology for the primary diagnosis of breast histopathological specimens: an innovative validation and concordance study on digital pathology validation and training , 2018, Histopathology.

[39]  Faliu Yi,et al.  Lung Cancer Pathological Image Analysis Using a Hidden Potts Model , 2017, Cancer informatics.

[40]  Ce Zhang,et al.  Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features , 2016, Nature Communications.

[41]  Rafael Sirera,et al.  The Role of Tumor-Infiltrating Immune Cells and Chronic Inflammation at the Tumor Site on Cancer Development, Progression, and Prognosis: Emphasis on Non-small Cell Lung Cancer , 2011, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[42]  Andrew D Althouse,et al.  Accuracy of the IASLC/ATS/ERS histological subtyping of stage I lung adenocarcinoma on intraoperative frozen sections , 2015, Modern Pathology.

[43]  Junzhou Huang,et al.  Automatic extraction of cell nuclei from H&E-stained histopathological images , 2017, Journal of medical imaging.

[44]  Yang Xie,et al.  Artificial Intelligence in Lung Cancer Pathology Image Analysis , 2019, Cancers.

[45]  Masahiro Tsuboi,et al.  Abundant tumor promoting stromal cells in lung adenocarcinoma with hypoxic regions. , 2018, Lung cancer.

[46]  Erik Sahai,et al.  Mesenchymal Cancer Cell-Stroma Crosstalk Promotes Niche Activation, Epithelial Reversion, and Metastatic Colonization , 2015, Cell reports.

[47]  Thomas P. Buck,et al.  Validation of a whole slide imaging system for primary diagnosis in surgical pathology: A community hospital experience , 2014, Journal of pathology informatics.

[48]  Yee-Wah Tsang,et al.  Validation of digital pathology imaging for primary histopathological diagnosis , 2016, Histopathology.

[49]  Junzhou Huang,et al.  Comprehensive Computational Pathological Image Analysis Predicts Lung Cancer Prognosis , 2017, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[50]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[51]  F. Markowetz,et al.  Quantitative Image Analysis of Cellular Heterogeneity in Breast Tumors Complements Genomic Profiling , 2012, Science Translational Medicine.

[52]  Noriaki Ohuchi,et al.  Prognostic significance of tumor-infiltrating CD8+ and FOXP3+ lymphocytes in residual tumors and alterations in these parameters after neoadjuvant chemotherapy in triple-negative breast cancer: a retrospective multicenter study , 2015, Breast Cancer Research.

[53]  Rafael Sirera,et al.  The Role of Tumor Stroma in Cancer Progression and Prognosis: Emphasis on Carcinoma-Associated Fibroblasts and Non-small Cell Lung Cancer , 2011, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[54]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[55]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[56]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[57]  Andrew Janowczyk,et al.  Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images , 2017, Scientific Reports.

[58]  A. R. Jonckheere,et al.  A DISTRIBUTION-FREE k-SAMPLE TEST AGAINST ORDERED ALTERNATIVES , 1954 .

[59]  Gwénaël Le Teuff,et al.  Subtype Classification of Lung Adenocarcinoma Predicts Benefit From Adjuvant Chemotherapy in Patients Undergoing Complete Resection. , 2015, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[60]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[61]  Jae Hyuk Lee,et al.  Prognostic significance of tumor-infiltrating lymphocytes for patients with colorectal cancer. , 2012, Archives of surgery.

[62]  Oriol Ordi,et al.  Validation of whole slide imaging in the primary diagnosis of gynaecological pathology in a University Hospital , 2014, Journal of Clinical Pathology.

[63]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[64]  Andrew H. Beck,et al.  Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival , 2011, Science Translational Medicine.

[65]  Dipti Prasad Mukherjee,et al.  Mitosis Detection for Invasive Breast Cancer Grading in Histopathological Images , 2015, IEEE Transactions on Image Processing.

[66]  A. Gazdar,et al.  Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome , 2018, Scientific Reports.

[67]  D. Hanahan,et al.  Hallmarks of Cancer: The Next Generation , 2011, Cell.

[68]  P Waliszewski,et al.  [Fractal geometry in the objective grading of prostate carcinoma]. , 2014, Der Urologe. Ausg. A.

[69]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[70]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[71]  T. Dønnem,et al.  Prognostic Effect of Epithelial and Stromal Lymphocyte Infiltration in Non–Small Cell Lung Cancer , 2008, Clinical Cancer Research.

[72]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.