Pan-cancer image-based detection of clinically actionable genetic alterations

Precision treatment of cancer relies on genetic alterations which are diagnosed by molecular biology assays.1 These tests can be a bottleneck in oncology workflows because of high turnaround time, tissue usage and costs.2 Here, we show that deep learning can predict point mutations, molecular tumor subtypes and immune-related gene expression signatures3,4 directly from routine histological images of tumor tissue. We developed and systematically optimized a one-stop-shop workflow and applied it to more than 4000 patients with breast5, colon and rectal6, head and neck7, lung8,9, pancreatic10, prostate11 cancer, melanoma12 and gastric13 cancer. Together, our findings show that a single deep learning algorithm can predict clinically actionable alterations from routine histology data. Our method can be implemented on mobile hardware14, potentially enabling point-of-care diagnostics for personalized cancer treatment in individual patients.

[1]  H. Brenner,et al.  Long-term risk of colorectal cancer after negative colonoscopy. , 2011, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[2]  Steven J. M. Jones,et al.  Comprehensive molecular characterization of human colon and rectal cancer , 2012, Nature.

[3]  Steven J. M. Jones,et al.  Comprehensive genomic characterization of squamous cell lung cancers , 2012, Nature.

[4]  Steven J. M. Jones,et al.  Comprehensive molecular portraits of human breast tumors , 2012, Nature.

[5]  Steven J. M. Jones,et al.  Comprehensive molecular portraits of human breast tumours , 2013 .

[6]  Steven J. M. Jones,et al.  Comprehensive molecular characterization of gastric adenocarcinoma , 2014, Nature.

[7]  A. Egloff,et al.  Caspase‐8 mutations in head and neck cancer confer resistance to death receptor‐mediated apoptosis and enhance migration, invasion, and tumor growth , 2014, Molecular oncology.

[8]  Steven J. M. Jones,et al.  Comprehensive molecular profiling of lung adenocarcinoma , 2014, Nature.

[9]  Jeffrey S. Morris,et al.  The Consensus Molecular Subtypes of Colorectal Cancer , 2015, Nature Medicine.

[10]  Steven J. M. Jones,et al.  Genomic Classification of Cutaneous Melanoma , 2015, Cell.

[11]  M. Kloor,et al.  Statin use and survival after colorectal cancer: the importance of comprehensive confounder adjustment. , 2015, Journal of the National Cancer Institute.

[12]  Steven J. M. Jones,et al.  Comprehensive genomic characterization of head and neck squamous cell carcinomas , 2015, Nature.

[13]  Kyung-Ja Cho,et al.  Epithelial-Mesenchymal Transition , 2014, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[14]  Steven J. M. Jones,et al.  The Molecular Taxonomy of Primary Prostate Cancer , 2015, Cell.

[15]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[18]  Steven J. M. Jones,et al.  Integrated Genomic Characterization of Pancreatic Ductal Adenocarcinoma. , 2017, Cancer cell.

[19]  Wenlin Huang,et al.  CDC27 Induces Metastasis and Invasion in Colorectal Cancer via the Promotion of Epithelial-To-Mesenchymal Transition , 2017, Journal of Cancer.

[20]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  A. Chinnaiyan,et al.  Inactivation of CDK12 Delineates a Distinct Immunogenic Class of Advanced Prostate Cancer , 2018, Cell.

[22]  Chandra Sekhar Pedamallu,et al.  Comparative Molecular Analysis of Gastrointestinal Adenocarcinomas. , 2018, Cancer cell.

[23]  M. Berger,et al.  Clinical tumour sequencing for precision oncology: time for a universal strategy , 2018, Nature Reviews Cancer.

[24]  Xiaotu Ma,et al.  Clinical cancer genomic profiling by three-platform sequencing of whole genome, whole exome and transcriptome , 2018, Nature Communications.

[25]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[26]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Andrew J. Schaumberg,et al.  D R A F T H&E-stained Whole Slide Image Deep Learning Predicts SPOP Mutation State in Prostate Cancer , 2017 .

[28]  Evert Bosdriesz,et al.  MAP3K1 and MAP2K4 mutations are associated with sensitivity to MEK inhibitors in multiple cancer models , 2018, Cell Research.

[29]  Steven J. M. Jones,et al.  The Immune Landscape of Cancer , 2018, Immunity.

[30]  Timo Kohlberger,et al.  Microscope 2.0: An Augmented Reality Microscope with Real-time Artificial Intelligence Integration , 2018, ArXiv.

[31]  Jakob Nikolas Kather,et al.  Genomics and emerging biomarkers for immunotherapy of colorectal cancer. , 2018, Seminars in cancer biology.

[32]  Jakob Nikolas Kather,et al.  Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer , 2019, Nature Medicine.

[33]  Jakob Nikolas Kather,et al.  Deep learning detects virus presence in cancer histology , 2019, bioRxiv.

[34]  Timo Kohlberger,et al.  An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis , 2019, Nature Medicine.

[35]  N. Razavian,et al.  A Deep Learning Approach for Rapid Mutational Screening in Melanoma , 2019, bioRxiv.

[36]  H. Rugo,et al.  Alpelisib for PIK3CA‐Mutated, Hormone Receptor–Positive Advanced Breast Cancer , 2019, The New England journal of medicine.

[37]  K. Kojima,et al.  A subset of diffuse-type gastric cancer is susceptible to mTOR inhibitors and checkpoint inhibitors , 2019, Journal of Experimental & Clinical Cancer Research.

[38]  Constantino Carlos Reyes-Aldasoro,et al.  Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study , 2019, PLoS medicine.

[39]  Thomas J. Fuchs,et al.  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images , 2019, Nature Medicine.