Deep learning identifies inflamed fat as a risk factor for lymph node metastasis in early colorectal cancer

The spread of early‐stage (T1 and T2) adenocarcinomas to locoregional lymph nodes is a key event in disease progression of colorectal cancer (CRC). The cellular mechanisms behind this event are not completely understood and existing predictive biomarkers are imperfect. Here, we used an end‐to‐end deep learning algorithm to identify risk factors for lymph node metastasis (LNM) status in digitized histopathology slides of the primary CRC and its surrounding tissue. In two large population‐based cohorts, we show that this system can predict the presence of more than one LNM in pT2 CRC patients with an area under the receiver operating curve (AUROC) of 0.733 (0.67–0.758) and patients with any LNM with an AUROC of 0.711 (0.597–0.797). Similarly, in pT1 CRC patients, the presence of more than one LNM or any LNM was predictable with an AUROC of 0.733 (0.644–0.778) and 0.567 (0.542–0.597), respectively. Based on these findings, we used the deep learning system to guide human pathology experts towards highly predictive regions for LNM in the whole slide images. This hybrid human observer and deep learning approach identified inflamed adipose tissue as the highest predictive feature for LNM presence. Our study is a first proof of concept that artificial intelligence (AI) systems may be able to discover potentially new biological mechanisms in cancer progression. Our deep learning algorithm is publicly available and can be used for biomarker discovery in any disease setting. © 2021 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

[1]  Jakob Nikolas Kather,et al.  Weakly supervised annotation‐free cancer detection and prediction of genotype in routine histopathology , 2021, The Journal of pathology.

[2]  Jakob Nikolas Kather,et al.  Development and validation of deep learning classifiers to detect Epstein-Barr virus and microsatellite instability status in gastric cancer: a retrospective multicentre cohort study , 2021, The Lancet. Digital health.

[3]  Jakob Nikolas Kather,et al.  The impact of site-specific digital histology signatures on deep learning model accuracy and bias , 2021, Nature Communications.

[4]  Jakob Nikolas Kather,et al.  Artificial Intelligence-based Detection of FGFR3 Mutational Status Directly from Routine Histology in Bladder Cancer: A Possible Preselection for Molecular Testing? , 2021, European urology focus.

[5]  C. Desmedt,et al.  Breast adipocyte size associates with ipsilateral invasive breast cancer risk after ductal carcinoma in situ , 2021, NPJ breast cancer.

[6]  R. Maron,et al.  Deep learning approach to predict lymph node metastasis directly from primary tumour histology in prostate cancer , 2021, BJU international.

[7]  Masaru Ishii,et al.  Morphological and molecular breast cancer profiling through explainable machine learning , 2021, Nature Machine Intelligence.

[8]  B. Martin,et al.  Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. , 2021, The Lancet. Oncology.

[9]  S. Kudo,et al.  Risk Stratification of T1 Colorectal Cancer Metastasis to Lymph Nodes: Current Status and Perspective , 2020, Gut and liver.

[10]  Jakob Nikolas Kather,et al.  Artificial intelligence-based pathology for gastrointestinal and hepatobiliary cancers , 2020, Gut.

[11]  Jakob Nikolas Kather,et al.  Deep learning in cancer pathology: a new generation of clinical biomarkers , 2020, British Journal of Cancer.

[12]  Ellery Wulczyn,et al.  Interpretable survival prediction for colorectal cancer using deep learning , 2020, npj Digital Medicine.

[13]  D. Treanor,et al.  Guidelines for clinical trials using artificial intelligence – SPIRIT‐AI and CONSORT‐AI† , 2020, The Journal of pathology.

[14]  Pierre Courtiol,et al.  A deep learning model to predict RNA-Seq expression of tumours from whole slide images , 2020, Nature Communications.

[15]  Jakob Nikolas Kather,et al.  Development of AI-based pathology biomarkers in gastrointestinal and liver cancer , 2020, Nature Reviews Gastroenterology & Hepatology.

[16]  Jeroen van der Laak,et al.  HookNet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images , 2020, Medical Image Anal..

[17]  Alexander T. Pearson,et al.  Clinical-grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning. , 2020, Gastroenterology.

[18]  Fan Yang,et al.  Predicting Lymph Node Metastasis Using Histopathological Images Based on Multiple Instance Learning With Deep Graph Convolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jakob Nikolas Kather,et al.  The Aachen Protocol for Deep Learning Histopathology: A hands-on guide for data preprocessing , 2020 .

[20]  Marco Novelli,et al.  Deep learning for prediction of colorectal cancer outcome: a discovery and validation study , 2020, The Lancet.

[21]  Jakob Nikolas Kather,et al.  Pan-cancer image-based detection of clinically actionable genetic alterations , 2019, Nature Cancer.

[22]  Alexander W. Jung,et al.  Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis , 2019, Nature Cancer.

[23]  Shoichi Saito,et al.  Prediction of early colorectal cancer metastasis by machine learning using digital slide images , 2019, Comput. Methods Programs Biomed..

[24]  Jakob Nikolas Kather,et al.  Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer , 2019, Nature Medicine.

[25]  P. Stepnowski,et al.  Changes in lipids composition and metabolism in colorectal cancer: a review , 2019, Lipids in Health and Disease.

[26]  Constantino Carlos Reyes-Aldasoro,et al.  Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study , 2019, PLoS medicine.

[27]  T. Bisgaard,et al.  Malignant colorectal polyps: endoscopic polypectomy and watchful waiting is not inferior to subsequent bowel resection. A nationwide propensity score-based analysis , 2018, Langenbeck's Archives of Surgery.

[28]  Jakob Nikolas Kather,et al.  Automatic evaluation of tumor budding in immunohistochemically stained colorectal carcinomas and correlation to clinical outcome , 2018, Diagnostic Pathology.

[29]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[30]  E. Lengyel,et al.  Adipocyte-induced CD36 expression drives ovarian cancer progression and metastasis , 2018, Oncogene.

[31]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[32]  A. Ramírez de Molina,et al.  Complementary ACSL isoforms contribute to a non-Warburg advantageous energetic status characterizing invasive colon cancer cells , 2017, Scientific Reports.

[33]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Kaisa Liimatainen,et al.  Metastasis detection from whole slide images using local features and random forests , 2017, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[35]  Peter Bankhead,et al.  QuPath: Open source software for digital pathology image analysis , 2017, Scientific Reports.

[36]  S. Pritchard,et al.  Interobserver variability amongst gastrointestinal pathologists in assessing prognostic parameters of malignant colorectal polyps: a cause for concern , 2016, Techniques in Coloproctology.

[37]  Ahmedin Jemal,et al.  Global patterns and trends in colorectal cancer incidence and mortality , 2016, Gut.

[38]  Mihail I Mitov,et al.  Increased expression of fatty acid synthase provides a survival advantage to colorectal cancer cells via upregulation of cellular respiration , 2015, Oncotarget.

[39]  P. Quirke,et al.  Area of Submucosal Invasion and Width of Invasion Predicts Lymph Node Metastasis in pT1 Colorectal Cancers , 2015, Diseases of the colon and rectum.

[40]  Gary S Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement , 2015, BMC Medicine.

[41]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[42]  Robert V Farese,et al.  Cellular fatty acid metabolism and cancer. , 2013, Cell metabolism.

[43]  I. Nagtegaal,et al.  Predicting lymph node metastasis in pT1 colorectal cancer: a systematic review of risk factors providing rationale for therapy decisions , 2013, Endoscopy.

[44]  H. Ueno,et al.  Objective criteria for crohn-like lymphoid reaction in colorectal cancer. , 2013, American journal of clinical pathology.

[45]  R. Steele,et al.  Impact of the UK colorectal cancer screening pilot studies on incidence, stage distribution and mortality trends. , 2012, Cancer epidemiology.

[46]  G. Mills,et al.  Adipocytes promote ovarian cancer metastasis and provide energy for rapid tumor growth , 2011, Nature Medicine.

[47]  Filip Lardon,et al.  A review of the most promising biomarkers in colorectal cancer: one step closer to targeted therapy. , 2010, The oncologist.

[48]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[49]  S. Gruber,et al.  Pathologic Predictors of Microsatellite Instability in Colorectal Cancer , 2009, The American journal of surgical pathology.

[50]  A. Harris,et al.  REporting recommendations for tumour MARKer prognostic studies (REMARK) , 2005, British Journal of Cancer.

[51]  K. Batts,et al.  Interobserver variability in the pathological assessment of malignant colorectal polyps , 2004, The British journal of surgery.

[52]  J. Hardcastle,et al.  Colorectal cancer , 1993, Europe Against Cancer European Commission Series for General Practitioners.