A Means of Assessing Deep Learning-Based Detection of ICOS Protein Expression in Colon Cancer

Simple Summary In this study, we propose a general artificial intelligence (AI) based workflow for applying deep learning to the problem of cell identification in immunohistochemistry-stained slides as a basis for quantifying nuclear staining biomarkers. Our approach consists of two main parts: a simplified but robust annotation process, and the application of cell identification models. This results in an optimised process with a new user-friendly tool that can interact with other open-source software and assists pathologists and scientists in creating and exporting data for deep learning. We present a set of architectures for cell identification to quantify and analyse the trade-offs between different deep learning architectures, providing a more accurate and less time-consuming tool than using traditional methods. This approach can identify the best tool to deliver AI tools for clinical utility. Abstract Biomarkers identify patient response to therapy. The potential immune-checkpoint biomarker, Inducible T-cell COStimulator (ICOS), expressed on regulating T-cell activation and involved in adaptive immune responses, is of great interest. We have previously shown that open-source software for digital pathology image analysis can be used to detect and quantify ICOS using cell detection algorithms based on traditional image processing techniques. Currently, artificial intelligence (AI) based on deep learning methods is significantly impacting the domain of digital pathology, including the quantification of biomarkers. In this study, we propose a general AI-based workflow for applying deep learning to the problem of cell segmentation/detection in IHC slides as a basis for quantifying nuclear staining biomarkers, such as ICOS. It consists of two main parts: a simplified but robust annotation process, and cell segmentation/detection models. This results in an optimised annotation process with a new user-friendly tool that can interact with1 other open-source software and assists pathologists and scientists in creating and exporting data for deep learning. We present a set of architectures for cell-based segmentation/detection to quantify and analyse the trade-offs between them, proving to be more accurate and less time consuming than traditional methods. This approach can identify the best tool to deliver the prognostic significance of ICOS protein expression.

[1]  Yinyin Yuan,et al.  Artificial intelligence and digital pathology: Opportunities and implications for immuno-oncology , 2021, Biochimica et biophysica acta. Reviews on cancer.

[2]  Kris D. McCombe,et al.  Immune status is prognostic for poor survival in colorectal cancer patients and is associated with tumour hypoxia , 2020, British Journal of Cancer.

[3]  Jun Ma,et al.  Segmentation Loss Odyssey , 2020, ArXiv.

[4]  M. Rugge,et al.  Gastrointestinal tissue‐based molecular biomarkers: a practical categorisation based on the 2019 World Health Organization classification of epithelial digestive tumours , 2020, Histopathology.

[5]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[7]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[8]  Geert J. S. Litjens,et al.  Learning to detect lymphocytes in immunohistochemistry with deep learning , 2019, Medical Image Anal..

[9]  Hao Xu,et al.  Analysis of the Human Protein Atlas Image Classification competition , 2019, Nature Methods.

[10]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[11]  M. Salto‐Tellez,et al.  Critical Appraisal of Programmed Death Ligand 1 Reflex Diagnostic Testing: Current Standards and Future Opportunities , 2019, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[12]  Horst K. Hahn,et al.  Deep learning nuclei detection: A simple approach can deliver state-of-the-art results , 2018, Comput. Medical Imaging Graph..

[13]  Casper F Winsnes,et al.  Deep learning is combined with massive-scale citizen science to improve large-scale image classification , 2018, Nature Biotechnology.

[14]  Nasir M. Rajpoot,et al.  A Multi-resolution Deep Learning Framework for Lung Adenocarcinoma Growth Pattern Classification , 2018, MIUA.

[15]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[16]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Chandan Chakraborty,et al.  Efficient deep learning model for mitosis detection using breast histopathology images , 2017, Comput. Medical Imaging Graph..

[18]  M. Salto‐Tellez,et al.  Standardising RNA profiling based biomarker application in cancer-The need for robust control of technical variables. , 2017, Biochimica et biophysica acta. Reviews on cancer.

[19]  A. Gavin,et al.  Evaluation of PTGS2 Expression, PIK3CA Mutation, Aspirin Use and Colon Cancer Survival in a Population-Based Cohort Study , 2017, Clinical and Translational Gastroenterology.

[20]  A. Gavin,et al.  Statin use, candidate mevalonate pathway biomarkers, and colon cancer survival in a population-based cohort study , 2017, British Journal of Cancer.

[21]  Yoshua Bengio,et al.  A robust adaptive stochastic gradient method for deep learning , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[22]  M. Salto‐Tellez,et al.  Back to the future: routine morphological assessment of the tumour microenvironment is prognostic in stage II/III colon cancer in a large population‐based study , 2017, Histopathology.

[23]  Manuel Salto-Tellez,et al.  Tissue-based next generation sequencing: application in a universal healthcare system , 2017, British Journal of Cancer.

[24]  Peter Bankhead,et al.  QuPath: Open source software for digital pathology image analysis , 2017, Scientific Reports.

[25]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[28]  M. Salto‐Tellez,et al.  Building a 'Repository of Science': The importance of integrating biobanks within molecular pathology programmes. , 2016, European journal of cancer.

[29]  M. Salto‐Tellez,et al.  MErCuRIC1: A phase 1a study of MEK1/2 inhibitor PD-0325901 with cMET inhibitor crizotinib in patients with advanced solid tumours , 2016 .

[30]  M. Salto‐Tellez,et al.  Validation of immunocytochemistry as a morphomolecular technique , 2016, Cancer cytopathology.

[31]  M. Salto‐Tellez,et al.  Abstract 924: Upregulation of the MET transcript is consistently associated with invasion and tumor budding in colorectal cancer , 2016 .

[32]  M. Salto‐Tellez,et al.  Caspase modelling to predict personalised risk in stage III colorectal cancer (CRC) patients. , 2016 .

[33]  M. Salto‐Tellez,et al.  A systems model of BCL-2 dependent apoptosis to predict stage II CRC patients benefiting from adjuvant chemotherapy and as a prognostic tool for stage III CRC patients with increased risk of recurrence. , 2016 .

[34]  Mark Lawler,et al.  Challenging the Cancer Molecular Stratification Dogma: Intratumoral Heterogeneity Undermines Consensus Molecular Subtypes and Potential Diagnostic Value in Colorectal Cancer , 2016, Clinical Cancer Research.

[35]  M. Salto‐Tellez,et al.  Stratified analysis reveals chemokine-like factor (CKLF) as a potential prognostic marker in the MSI-immune consensus molecular subtype CMS1 of colorectal cancer , 2016, Oncotarget.

[36]  P. Maxwell,et al.  Abstract 1396: Validation of a MEK/MET-specific NGS panel for early phase trial interrogation , 2016 .

[37]  James R. Eshleman,et al.  Microsatellite Instability as a Biomarker for PD-1 Blockade , 2016, Clinical Cancer Research.

[38]  Ahmedin Jemal,et al.  Global patterns and trends in colorectal cancer incidence and mortality , 2016, Gut.

[39]  M. Salto‐Tellez,et al.  PTEN deficiency promotes macrophage infiltration and hypersensitivity of prostate cancer to IAP antagonist/radiation combination therapy , 2016, Oncotarget.

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  M. Salto‐Tellez,et al.  Molecular classification of non-invasive breast lesions for personalised therapy and chemoprevention , 2015, Oncotarget.

[42]  M. Salto‐Tellez,et al.  EphA2 Expression Is a Key Driver of Migration and Invasion and a Poor Prognostic Marker in Colorectal Cancer , 2015, Clinical Cancer Research.

[43]  Yinhai Wang,et al.  Automated tumor analysis for molecular profiling in lung cancer , 2015, Oncotarget.

[44]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[45]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[48]  D. Rimm,et al.  Prognostic Biomarkers in Phase II Trial of Cetuximab-Containing Induction and Chemoradiation in Resectable HNSCC: Eastern Cooperative Oncology Group E2303 , 2014, Clinical Cancer Research.

[49]  C. Rödel,et al.  Tumour-infiltrating lymphocytes predict response to definitive chemoradiotherapy in head and neck cancer , 2013, British Journal of Cancer.

[50]  H. Grabsch,et al.  Guidelines and considerations for conducting experiments using tissue microarrays , 2013, Histopathology.

[51]  Manuel Salto-Tellez,et al.  Gelsolin Induces Colorectal Tumor Cell Invasion via Modulation of the Urokinase-Type Plasminogen Activator Cascade , 2012, PloS one.

[52]  Arthur T. Johnson,et al.  Quantitative comparison of immunohistochemical staining measured by digital image analysis versus pathologist visual scoring , 2012, Diagnostic Pathology.

[53]  J. Väyrynen,et al.  An improved image analysis method for cell counting lends credibility to the prognostic significance of T cells in colorectal cancer , 2012, Virchows Archiv.

[54]  Kok Kiong Tan,et al.  Development of a Portable Tissue Micro Array Instrument , 2011 .

[55]  M. Salto‐Tellez,et al.  Overexpression of neurone glial‐related cell adhesion molecule is an independent predictor of poor prognosis in advanced colorectal cancer , 2011, Cancer science.

[56]  M. Salto‐Tellez,et al.  Elevated NRD1 metalloprotease expression plays a role in breast cancer growth and proliferation , 2011, Genes, chromosomes & cancer.

[57]  Jie Yang,et al.  Clinical and Testing Protocols for the Analysis of Epidermal Growth Factor Receptor Mutations in East Asian Patients with Non-small Cell Lung Cancer: A Combined Clinical-Molecular Pathological Approach , 2011, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[58]  Kol Jia Yong,et al.  Dual-colour HER2/chromosome 17 chromogenic in situ hybridisation enables accurate assessment of HER2 genomic status in ovarian tumours , 2011, Journal of Clinical Pathology.

[59]  Axel Benner,et al.  Localization and density of immune cells in the invasive margin of human colorectal cancer liver metastases are prognostic for response to chemotherapy. , 2011, Cancer research.

[60]  Kol Jia Yong,et al.  Dual-colour HER2/Chromosome 17 chromogenic in situ hybridisation assay enables accurate assessment of HER2 genomic status in gastric cancer and has potential utility in HER2 testing of biopsy samples , 2011, Journal of Clinical Pathology.

[61]  M. Salto‐Tellez,et al.  Where and by whom should gastric cancer HER2/neu status be assessed?: lessons from breast cancer. , 2011, Archives of pathology & laboratory medicine.

[62]  L. Wong,et al.  Low cytosine triphosphate synthase 2 expression renders resistance to 5-fluorouracil in colorectal cancer , 2011, Cancer biology & therapy.

[63]  M. Salto‐Tellez,et al.  Activated oncogenic pathways and therapeutic targets in extranodal nasal‐type NK/T cell lymphoma revealed by gene expression profiling , 2011, The Journal of pathology.

[64]  M. Salto‐Tellez,et al.  Molecular-assisted immunohistochemical optimization. , 2010, Acta histochemica.

[65]  M. Salto‐Tellez,et al.  Cancer-related ectopic expression of the bone-related transcription factor RUNX2 in non-osseous metastatic tumor cells is linked to cell proliferation and motility , 2010, Breast Cancer Research.

[66]  M. Salto‐Tellez,et al.  Aurora‐A expression, hormone receptor status and clinical outcome in hormone related cancers , 2010, Pathology.

[67]  M. Salto‐Tellez,et al.  Differential expression of steroid 5alpha-reductase isozymes and association with disease severity and angiogenic genes predict their biological role in prostate cancer. , 2010, Endocrine-related cancer.

[68]  M. Salto‐Tellez,et al.  MicroRNA-130b regulates the tumour suppressor RUNX3 in gastric cancer. , 2010, European journal of cancer.

[69]  M. Salto‐Tellez,et al.  Computer‐assisted pathological immunohistochemistry scoring is more time‐effective than conventional scoring, but provides no analytical advantage , 2010, Histopathology.

[70]  M. Salto‐Tellez,et al.  PIM‐1 kinase expression in adipocytic neoplasms: diagnostic and biological implications , 2010, International journal of experimental pathology.

[71]  M. Salto‐Tellez,et al.  Molecular Pathology in Contemporary Diagnostic Pathology Laboratory: An Opinion for the Active Role of Surgical Pathologists , 2010, The American journal of surgical pathology.

[72]  Mohd Feroz Mohd Omar,et al.  Cofilin immunolabelling correlates with depth of invasion in gastrointestinal endocrine cell tumors. , 2010, Acta histochemica.

[73]  M. Salto‐Tellez,et al.  Recurrent lower gastrointestinal bleeding secondary to cytomegalovirus‐associated colonic ulcer in a non human immunodeficiency virus infected patient: timely diagnosis and treatment averted surgery , 2009, Colorectal disease : the official journal of the Association of Coloproctology of Great Britain and Ireland.

[74]  M. Salto‐Tellez,et al.  Comparative validation of c‐kit exon 11 mutation analysis on cytology samples and corresponding surgical resections of gastrointestinal stromal tumours , 2009, Cytopathology : official journal of the British Society for Clinical Cytology.

[75]  M. Salto‐Tellez,et al.  Elucidating the molecular physiopathology of acute respiratory distress syndrome in severe acute respiratory syndrome patients , 2009, Virus Research.

[76]  M. Salto‐Tellez,et al.  GYY4137, a novel hydrogen sulfide-releasing molecule, protects against endotoxic shock in the rat. , 2009, Free radical biology & medicine.

[77]  M. Salto‐Tellez,et al.  TRIP-Br2 promotes oncogenesis in nude mice and is frequently overexpressed in multiple human tumors , 2009, Journal of Translational Medicine.

[78]  M. Salto‐Tellez,et al.  Denaturing High Performance Liquid Chromatography for the Detection of Microsatellite Instability Using Bethesda and Pentaplex Marker Panels , 2008, Diagnostic molecular pathology : the American journal of surgical pathology, part B.

[79]  M. Salto‐Tellez,et al.  TRARESA: a tissue microarray‐based hospital system for biomarker validation and discovery , 2008, Pathology.

[80]  M. Salto‐Tellez,et al.  Investigating the roles of gelsolin in the malignant progression of colorectal tumor cells , 2008 .

[81]  M. Salto‐Tellez,et al.  Bevacizumab and rapamycin induce growth suppression in mouse models of hepatocellular carcinoma. , 2008, Journal of hepatology.

[82]  M. Salto‐Tellez,et al.  Prognostic significance of thymidylate synthase, dihydropyrimidine dehydrogenase and thymidine phosphorylase protein expression in colorectal cancer patients treated with or without 5-fluorouracil-based chemotherapy. , 2008, Annals of oncology : official journal of the European Society for Medical Oncology.

[83]  M. Salto‐Tellez,et al.  932 Lymphocytic Follicles and Aggregates in Colonic Biopsies with Non-Specific Changes Are Related to Duration of Diarrhea , 2008 .

[84]  M. Salto‐Tellez,et al.  A Randomized crossover study comparing video endoscopic autofluorescence imaging followed by narrow band imaging with standard endoscopy for the detection of high risk lesions in stomach , 2008 .

[85]  Keli Ou,et al.  Novel breast cancer biomarkers identified by integrative proteomic and gene expression mapping. , 2008, Journal of proteome research.

[86]  M. Salto‐Tellez,et al.  Inactivation of RUNX3 by frequent promoter hypermethylation and protein mislocalization constitute an early event in breast cancer progression , 2008 .

[87]  M. Salto‐Tellez,et al.  RUNX3 inactivation by frequent promoter hypermethylation and protein mislocalization constitute an early event in breast cancer progression , 2008, Breast Cancer Research and Treatment.

[88]  M. Salto‐Tellez,et al.  Progressive loss of epidermal growth factor receptor in a subpopulation of breast cancers: implications in target-directed therapeutics , 2007, Molecular Cancer Therapeutics.

[89]  M. Salto‐Tellez,et al.  How do we encourage clinician scientists in Singapore? , 2007, Annals of the Academy of Medicine, Singapore.

[90]  M. Salto‐Tellez,et al.  Transplantation of encapsulated embryonic progenitor-derived insulin-producing cells reverses hyperglycemia in immune-competent diabetic animals , 2007 .

[91]  M. Salto‐Tellez,et al.  Differential expression of hDAB2IPA and hDAB2IPB in normal tissues and promoter methylation of hDAB2IPA in hepatocellular carcinoma. , 2007, Journal of hepatology.

[92]  M. Salto‐Tellez,et al.  Tissue microarrays characterise the clinical significance of a VEGF-A protein expression signature in gastrointestinal stromal tumours , 2007, British Journal of Cancer.

[93]  M. Salto‐Tellez,et al.  279 POSTER Pharmacodynamic effects of seliciclib (r-roscovitine, cyc202) in patients with undifferentiated nasopharyngeal cancer (NPC) using a window trial design , 2006 .

[94]  M. Salto‐Tellez,et al.  Prognostic and predictive significance of 5-fluorouracil metabolic enzymes in colorectal cancer. , 2006, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[95]  M. Salto‐Tellez,et al.  Mutations in sodium-borate cotransporter SLC4A11 cause recessive congenital hereditary endothelial dystrophy (CHED2) , 2006, Nature Genetics.

[96]  M. Salto‐Tellez,et al.  Insulin-producing cells derived from embryonic progenitor cells reverse hyperglycemia in streptozotocin-induced diabetic animals , 2006 .

[97]  M. Salto‐Tellez,et al.  Metastasis‐associated protein S100A4—a potential prognostic marker for colorectal cancer , 2006, Journal of surgical oncology.

[98]  M. Salto‐Tellez,et al.  Sensitive and cost-effective detection of epidermal growth factor receptor mutations in small biopsies by denaturing high performance liquid chromatography. , 2005 .

[99]  M. Salto‐Tellez,et al.  Pharmacodynamic effects of seliciclib (R-roscovitine, CYC202) in patients with undifferentiated nasopharyngeal carcinoma (NPC) using a window trial design. , 2005 .

[100]  M. Salto‐Tellez,et al.  Discordant quantitative detection of putative biomarkers in nodal micrometastases of colorectal cancer: biological and clinical implications , 2005, Journal of Clinical Pathology.

[101]  M. Salto‐Tellez,et al.  Confocal laser endoscopy is useful for in-vivo rapid diagnosis of gastric neoplasia and pre-neoplasia , 2005 .

[102]  M. Salto‐Tellez,et al.  Inhibition of histone deacetylase 2 increases apoptosis and p21Cip1/WAF1 expression, independent of histone deacetylase 1 , 2005, Cell Death and Differentiation.

[103]  M. Salto‐Tellez,et al.  Metastatic Testicular Carcinoma From the Colon With Clinical, Immunophenotypical, and Molecular Characterization: Report of a Case , 2005, Diseases of the colon and rectum.

[104]  M. Salto‐Tellez,et al.  Transplantation of neonatal porcine islets and sertoli cells into nonimmunosuppressed nonhuman primates. , 2004, Transplantation proceedings.

[105]  M. Salto‐Tellez,et al.  A model for directed embryonic stem cell differentiation: Establishing lineage-restricted progenitor cell lines , 2005 .

[106]  M. Salto‐Tellez,et al.  Xenotransplantation of neonatal porcine islets and Sertoli cells into nonimmunosuppressed streptozotocin-induced diabetic rats. , 2005, Transplantation proceedings.

[107]  M. Salto‐Tellez,et al.  T-cell post-transplant lymphoproliferative disorder after hematopoietic stem cell transplantation: another case and a review of the literature , 2004, Bone Marrow Transplantation.

[108]  M. Salto‐Tellez,et al.  Lymphocytic gastritis-like T cell lymphoma: molecular evidence of an unusual recurrence , 2004, Journal of Clinical Pathology.

[109]  M. Salto‐Tellez,et al.  Microsatellite instability in colorectal cancer: considerations for molecular diagnosis and high-throughput screening of archival tissues. , 2004, Clinical chemistry.

[110]  M. Salto‐Tellez,et al.  Sequence confirmation of the EWS‐WT1 fusion gene transcript in the peritoneal effusion of a patient with desmoplastic small round cell tumor , 2003, Diagnostic cytopathology.

[111]  P. Nowell,et al.  Multiplex RT-PCR for the detection of leukemia-associated translocations: validation and application to routine molecular diagnostic practice. , 2003, The Journal of molecular diagnostics : JMD.

[112]  Ronald Simon,et al.  Tissue microarray (TMA) applications: implications for molecular medicine , 2003, Expert Reviews in Molecular Medicine.

[113]  M. Salto‐Tellez,et al.  Intrinsic variability in the detection of micrometastases in lymph nodes for re-staging of colorectal cancer. effect of individual markers and tissue samples. , 2003, European journal of cancer.

[114]  M. Salto‐Tellez,et al.  Diagnostic validation of capillary electrophoresis analysis of T-cell receptor gamma-chain gene rearrangements: prediction of malignant transformation of cutaneous T-cell lymphoproliferative disorders. , 2003, Clinical chemistry.

[115]  M. Salto‐Tellez,et al.  AFP+, ESC‐Derived Cells Engraft and Differentiate into Hepatocytes in Vivo , 2002, Stem cells.

[116]  M. Salto‐Tellez,et al.  Microsatellite instability and mutational status of human K-ras oncogene in hereditary non-polyposis colorectal cancer. , 2001 .

[117]  M. Salto‐Tellez,et al.  The effect of the biopsy number on sensitivity and specificity of ultra-rapid urease test in detecting Helicobacter pylori: A prospective study , 2001 .

[118]  N. Shepherd What is the significance of muciphages in colorectal biopsies? , 2000, Histopathology.

[119]  M. Salto‐Tellez,et al.  Diagnosis of multiple granular cell tumours by fine needle aspiration cytology. , 2000, Cytopathology : official journal of the British Society for Clinical Cytology.

[120]  J. Kononen,et al.  Tissue microarrays for high-throughput molecular profiling of tumor specimens , 1998, Nature Medicine.