Recent advances of pathomics in colorectal cancer diagnosis and prognosis

Colorectal cancer (CRC) is one of the most common malignancies, with the third highest incidence and the second highest mortality in the world. To improve the therapeutic outcome, the risk stratification and prognosis predictions would help guide clinical treatment decisions. Achieving these goals have been facilitated by the fast development of artificial intelligence (AI) -based algorithms using radiological and pathological data, in combination with genomic information. Among them, features extracted from pathological images, termed pathomics, are able to reflect sub-visual characteristics linking to better stratification and prediction of therapeutic responses. In this paper, we review recent advances in pathological image-based algorithms in CRC, focusing on diagnosis of benign and malignant lesions, micro-satellite instability, as well as prediction of neoadjuvant chemoradiotherapy and the prognosis of CRC patients.

[1]  M. Garg,et al.  341. GeneXpert Ultra for the Diagnosis of Extrapulmonary Lymphnode Tuberculosis – A Diagnostic Study from High Tuberculosis Burden Country , 2022, Open Forum Infectious Diseases.

[2]  P. Ding,et al.  Clinical actionability of triaging DNA mismatch repair deficient colorectal cancer from biopsy samples using deep learning , 2022, EBioMedicine.

[3]  Sharda Vashisth,et al.  A convolution neural network with multi-level convolutional and attention learning for classification of cancer grades and tissue structures in colon histopathological images , 2022, Comput. Biol. Medicine.

[4]  Dimitris N. Metaxas,et al.  DigestPath: A benchmark dataset with challenge review for the pathological detection and segmentation of digestive-system , 2022, Medical Image Anal..

[5]  A. Dekker,et al.  Automatic Tumor Grading on Colorectal Cancer Whole-Slide Images: Semi-Quantitative Gland Formation Percentage and New Indicator Exploration , 2022, Frontiers in Oncology.

[6]  E. Imyanitov,et al.  Deep Learning Analysis of the Adipose Tissue and the Prediction of Prognosis in Colorectal Cancer , 2022, Frontiers in Nutrition.

[7]  Ming Y. Lu,et al.  Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology , 2022, Medical Image Anal..

[8]  Sung Hak Lee,et al.  Spatial analysis of tumor‐infiltrating lymphocytes in histological sections using deep learning techniques predicts survival in colorectal carcinoma , 2022, The journal of pathology. Clinical research.

[9]  C.V. Jawahar,et al.  Exploring Histological Similarities Across Cancers From a Deep Learning Perspective , 2022, Frontiers in Oncology.

[10]  T. Brinker,et al.  Artificial intelligence for detection of microsatellite instability in colorectal cancer—a multicentric analysis of a pre-screening tool for clinical application , 2022, ESMO open.

[11]  J. Gong,et al.  Development of a novel combined nomogram model integrating deep learning-pathomics, radiomics and immunoscore to predict postoperative outcome of colorectal cancer lung metastasis patients , 2022, Journal of Hematology & Oncology.

[12]  M. Hung,et al.  Development and validation of a radiopathomics model to predict pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: a multicentre observational study. , 2022, The Lancet. Digital health.

[13]  R. J. Martis,et al.  Comparative Study of Classification of Histopathological Images , 2021, 2021 5th International Conference on Electrical, Electronics, Communication, Computer Technologies and Optimization Techniques (ICEECCOT).

[14]  N. Rajpoot,et al.  SAFRON: Stitching Across the Frontier Network for Generating Colorectal Cancer Histology Images , 2021, Medical Image Anal..

[15]  N. Rajpoot,et al.  Development and validation of a weakly supervised deep learning framework to predict the status of molecular pathways and key mutations in colorectal cancer from routine histology images: a retrospective study , 2021, The Lancet. Digital health.

[16]  Jeffrey H. Chuang,et al.  Deep learning trained on hematoxylin and eosin tumor region of Interest predicts HER2 status and trastuzumab treatment response in HER2+ breast cancer , 2021, Modern Pathology.

[17]  Mesut Toğaçar,et al.  Disease type detection in lung and colon cancer images using the complement approach of inefficient sets , 2021, Comput. Biol. Medicine.

[18]  Yongquan Yang,et al.  Deep learning-based predictive biomarker of pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer , 2021, Journal of Translational Medicine.

[19]  H. Horlings,et al.  DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer , 2021, Medical Image Anal..

[20]  R. Dahlstrom,et al.  Challenges and opportunities , 2021, Foundations of a Sustainable Economy.

[21]  E. Trucco,et al.  Using machine learning approaches for multi-omics data analysis: A review. , 2021, Biotechnology advances.

[22]  Ruijiang Li,et al.  Predicting treatment response from longitudinal images using multi-task deep learning , 2021, Nature Communications.

[23]  Hanlee P. Ji,et al.  A deep learning model for molecular label transfer that enables cancer cell identification from histopathology images , 2021, bioRxiv.

[24]  Hongwei Li,et al.  Predicting gastric cancer outcome from resected lymph node histopathology images using deep learning , 2021, Nature Communications.

[25]  Philip Quirke,et al.  Deep learning detects genetic alterations in cancer histology generated by adversarial networks , 2021, The Journal of pathology.

[26]  A. Jemal,et al.  Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries , 2021, CA: a cancer journal for clinicians.

[27]  Selim Aksoy,et al.  Self-Supervised Learning with Graph Neural Networks for Region of Interest Retrieval in Histopathology , 2021, 2020 25th International Conference on Pattern Recognition (ICPR).

[28]  B. Martin,et al.  Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. , 2021, The Lancet. Oncology.

[29]  Honghao Gao,et al.  A Deep Learning Approach for Colonoscopy Pathology WSI Analysis: Accurate Segmentation and Classification , 2020, IEEE Journal of Biomedical and Health Informatics.

[30]  Ellery Wulczyn,et al.  Interpretable survival prediction for colorectal cancer using deep learning , 2020, npj Digital Medicine.

[31]  R. Socher,et al.  Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains , 2020, Nature Communications.

[32]  Yanqi Huang,et al.  Artificial intelligence quantified tumour-stroma ratio is an independent predictor for overall survival in resectable colorectal cancer , 2020, EBioMedicine.

[33]  Lei Zhang,et al.  Exploring prognostic indicators in the pathological images of hepatocellular carcinoma based on deep learning , 2020, Gut.

[34]  Jie Tian,et al.  Multiparametric MRI and Whole Slide Image-Based Pretreatment Prediction of Pathological Response to Neoadjuvant Chemoradiotherapy in Rectal Cancer: A Multicenter Radiopathomic Study , 2020, Annals of Surgical Oncology.

[35]  Yiping Wang,et al.  Synthesis of diagnostic quality cancer pathology images by generative adversarial networks , 2020, The Journal of pathology.

[36]  Chao Xu,et al.  Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images , 2020, Nature Communications.

[37]  D. Shen,et al.  Artificial Intelligence Neuropathologist for Glioma Classification using Deep Learning on Hematoxylin and Eosin Stained Slide images and Molecular Markers. , 2020, Neuro-oncology.

[38]  J. Meyerhardt,et al.  NCCN Guidelines Insights: Rectal Cancer, Version 6.2020. , 2020, Journal of the National Comprehensive Cancer Network : JNCCN.

[39]  Erratum: Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. , 2020, CA: a cancer journal for clinicians.

[40]  Ming Y. Lu,et al.  AI-based pathology predicts origins for cancers of unknown primary , 2020, Nature.

[41]  Alexander T. Pearson,et al.  Clinical-grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning. , 2020, Gastroenterology.

[42]  Qijiang Mao,et al.  Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning , 2020, npj Precision Oncology.

[43]  Yanqi Huang,et al.  Predicting treatment response to neoadjuvant chemoradiotherapy in local advanced rectal cancer by biopsy digital pathology image features , 2020, Clinical and translational medicine.

[44]  Ruchika Gupta,et al.  Whole Slide Imaging (WSI) in Pathology: Current Perspectives and Future Directions , 2020, Journal of Digital Imaging.

[45]  Jin Tae Kwak,et al.  Unsupervised Tumor Characterization via Conditional Generative Adversarial Networks , 2020, IEEE Journal of Biomedical and Health Informatics.

[46]  J. Yun,et al.  Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence , 2020, bioRxiv.

[47]  A. Jemal,et al.  Colorectal cancer statistics, 2020 , 2020, CA: a cancer journal for clinicians.

[48]  Marco Novelli,et al.  Deep learning for prediction of colorectal cancer outcome: a discovery and validation study , 2020, The Lancet.

[49]  Jakob Nikolas Kather,et al.  Pan-cancer image-based detection of clinically actionable genetic alterations , 2019, Nature Cancer.

[50]  A. Bardelli,et al.  Adaptive mutability of colorectal cancers in response to targeted therapies , 2019, Science.

[51]  Alexander W. Jung,et al.  Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis , 2019, Nature Cancer.

[52]  F. D. de Sauvage,et al.  The great escape: tumour cell plasticity in resistance to targeted therapy , 2019, Nature Reviews Drug Discovery.

[53]  Gokhan Bilgin,et al.  Classification of Nuclei in Colon Cancer Images using Ensemble of Deep Learned Features , 2019, 2019 Medical Technologies Congress (TIPTEKNO).

[54]  Tonya S. King,et al.  Challenges with colorectal cancer staging: results of an international study , 2019, Modern Pathology.

[55]  A. Scarpa,et al.  ESMO recommendations on microsatellite instability testing for immunotherapy in cancer, and its relationship with PD-1/PD-L1 expression and tumour mutational burden: a systematic review-based approach. , 2019, Annals of oncology : official journal of the European Society for Medical Oncology.

[56]  Tahsin Kurc,et al.  The Emergence of Pathomics , 2019, Current Pathobiology Reports.

[57]  Jeffrey H. Chuang,et al.  Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images , 2019, Nature Communications.

[58]  Joel H. Saltz,et al.  Robust Histopathology Image Analysis: To Label or to Synthesize? , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  M. Gurcan,et al.  Digital pathology and artificial intelligence. , 2019, The Lancet. Oncology.

[60]  Loris Nanni,et al.  Bioimage Classification with Handcrafted and Learned Features , 2019, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[61]  Hai Su,et al.  Pathologist-level interpretable whole-slide cancer diagnosis with deep learning , 2019, Nat. Mach. Intell..

[62]  Constantino Carlos Reyes-Aldasoro,et al.  Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study , 2019, PLoS medicine.

[63]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[64]  M. Baretti,et al.  DNA mismatch repair in cancer , 2018, Pharmacology & therapeutics.

[65]  H. Woodrow,et al.  : A Review of the , 2018 .

[66]  Sang Won Yoon,et al.  Medical Image Synthesis with Generative Adversarial Networks for Tissue Recognition , 2018, 2018 IEEE International Conference on Healthcare Informatics (ICHI).

[67]  Yavar Bathaee,et al.  The Artificial Intelligence Black Box and the Failure of Intent and Causation , 2018 .

[68]  Ghassan Hamarneh,et al.  Adversarial Stain Transfer for Histopathology Image Analysis , 2018, IEEE Transactions on Medical Imaging.

[69]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[70]  David B. A. Epstein,et al.  Glandular Morphometrics for Objective Grading of Colorectal Adenocarcinoma Histology Images , 2017, Scientific Reports.

[71]  Hamid R. Tizhoosh,et al.  A comparative study of CNN, BoVW and LBP for classification of histopathological images , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[72]  Zhenchao Tang,et al.  Radiomics Analysis for Evaluation of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer , 2017, Clinical Cancer Research.

[73]  Lin Yang,et al.  MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  J. Monson,et al.  Surgery for Patients With Rectal Cancer—Time to Listen to the Patients and Recognize Reality , 2017, JAMA oncology.

[75]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Junzhou Huang,et al.  Comprehensive Computational Pathological Image Analysis Predicts Lung Cancer Prognosis , 2017, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[77]  Peter Bankhead,et al.  QuPath: Open source software for digital pathology image analysis , 2017, Scientific Reports.

[78]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[79]  Jay Shendure,et al.  Classification and characterization of microsatellite instability across 18 cancer types , 2016, Nature Medicine.

[80]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[81]  Ce Zhang,et al.  Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features , 2016, Nature Communications.

[82]  Patrik L. Ståhl,et al.  Visualization and analysis of gene expression in tissue sections by spatial transcriptomics , 2016, Science.

[83]  Chao Lu,et al.  Retrospective study , 2016, Medicine.

[84]  N. Rajpoot,et al.  A model of the spatial tumour heterogeneity in colorectal adenocarcinoma tissue , 2016, BMC Bioinformatics.

[85]  Francesco Bianconi,et al.  Multi-class texture analysis in colorectal cancer histology , 2016, Scientific Reports.

[86]  Hao Chen,et al.  Gland segmentation in colon histology images: The glas challenge contest , 2016, Medical Image Anal..

[87]  N. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[88]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[89]  Hanlee P. Ji,et al.  The Cancer Genome Atlas Clinical Explorer: a web and mobile interface for identifying clinical–genomic driver associations , 2015, Genome Medicine.

[90]  Hanlee P. Ji,et al.  The Cancer Genome Atlas Clinical Explorer: a web and mobile interface for identifying clinical–genomic driver associations , 2015, Genome Medicine.

[91]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[92]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[93]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[94]  Vishal Monga,et al.  Automated discrimination of lower and higher grade gliomas based on histopathological image analysis , 2015, Journal of pathology informatics.

[95]  Nasir M. Rajpoot,et al.  A Nonlinear Mapping Approach to Stain Normalization in Digital Histopathology Images Using Image-Specific Color Deconvolution , 2014, IEEE Transactions on Biomedical Engineering.

[96]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[97]  J Rittscher,et al.  Digitally adjusting chromogenic dye proportions in brightfield microscopy images , 2012, Journal of microscopy.

[98]  A. Batistatou,et al.  Molecular basis of colorectal cancer. , 2010, The New England journal of medicine.

[99]  M. Bertagnolli,et al.  Molecular origins of cancer: Molecular basis of colorectal cancer. , 2009, The New England journal of medicine.

[100]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[101]  John McCarthy,et al.  A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955 , 2006, AI Mag..

[102]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[103]  A. Ruifrok,et al.  Quantification of histochemical staining by color deconvolution. , 2001, Analytical and quantitative cytology and histology.

[104]  C C Compton,et al.  Updated protocol for the examination of specimens from patients with carcinomas of the colon and rectum, excluding carcinoid tumors, lymphomas, sarcomas, and tumors of the vermiform appendix: a basis for checklists. Cancer Committee. , 2000, Archives of pathology & laboratory medicine.

[105]  松本 隆,et al.  Deconvolution , 1997, Computer Vision, A Reference Guide.

[106]  Stephen J. McKenna,et al.  Structure Prediction for Gland Segmentation With Hand-Crafted and Deep Convolutional Features , 2018, IEEE Transactions on Medical Imaging.

[107]  Andrew H. Beck,et al.  Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies. , 2018, Modern pathology : an official journal of the United States and Canadian Academy of Pathology, Inc.

[108]  W. Frankel,et al.  Serosal surfaces, mucin pools, and deposits, Oh my: challenges in staging colorectal carcinoma , 2015, Modern Pathology.

[109]  Steven N. Durlauf,et al.  Current perspectives and future directions , 2006 .