Clinical-grade computational pathology using weakly supervised deep learning on whole slide images

The development of decision support systems for pathology and their deployment in clinical practice have been hindered by the need for large manually annotated datasets. To overcome this problem, we present a multiple instance learning-based deep learning system that uses only the reported diagnoses as labels for training, thereby avoiding expensive and time-consuming pixel-wise manual annotations. We evaluated this framework at scale on a dataset of 44,732 whole slide images from 15,187 patients without any form of data curation. Tests on prostate cancer, basal cell carcinoma and breast cancer metastases to axillary lymph nodes resulted in areas under the curve above 0.98 for all cancer types. Its clinical application would allow pathologists to exclude 65–75% of slides while retaining 100% sensitivity. Our results show that this system has the ability to train accurate classification models at unprecedented scale, laying the foundation for the deployment of computational decision support systems in clinical practice.A deep learning model trained on real-world digital pathology data achieves clinical performance in cancer diagnosis.

[1]  Thomas Hofmann,et al.  Multiple instance learning with generalized support vector machines , 2002, AAAI/IAAI.

[2]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[3]  Rebecca L. Siegel Mph,et al.  Cancer statistics, 2016 , 2016 .

[4]  B. van Ginneken,et al.  Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis , 2016, Scientific Reports.

[5]  Qi Zhang,et al.  EM-DD: An Improved Multiple-Instance Learning Technique , 2001, NIPS.

[6]  Joachim M. Buhmann,et al.  Computational Pathology Analysis of Tissue Microarrays Predicts Survival of Renal Clear Cell Carcinoma Patients , 2008, MICCAI.

[7]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Abhijit Guha Roy,et al.  Classifying histopathology whole-slides using fusion of decisions from deep convolutional network on a collection of random multi-views at multi-magnification , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[10]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[11]  Nico Karssemeijer,et al.  Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies , 2018, Modern Pathology.

[12]  D. Brat,et al.  Predicting cancer outcomes from histology and genomics using convolutional networks , 2017, Proceedings of the National Academy of Sciences.

[13]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[14]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Mahadev Satyanarayanan,et al.  OpenSlide: A vendor-neutral software foundation for digital pathology , 2013, Journal of pathology informatics.

[16]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[17]  Nakul Verma Learning from data with low intrinsic dimension , 2012 .

[18]  S. Feldman,et al.  Incidence Estimate of Nonmelanoma Skin Cancer (Keratinocyte Carcinomas) in the U.S. Population, 2012. , 2015, JAMA dermatology.

[19]  Ş. Özdamar,et al.  Intraobserver and interobserver reproducibility of WHO and Gleason histologic grading systems in prostatic adenocarcinomas , 2006, International Urology and Nephrology.

[20]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[21]  Joachim M. Buhmann,et al.  Computational Pathology: Challenges and Promises for Tissue Analysis , 2015, Comput. Medical Imaging Graph..

[22]  Brendan J. Frey,et al.  Classifying and segmenting microscopy images with deep multiple instance learning , 2015, Bioinform..

[23]  A. Jemal,et al.  Cancer statistics, 2016 , 2016, CA: a cancer journal for clinicians.

[24]  J Carpenter,et al.  Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. , 2000, Statistics in medicine.

[25]  Yue Yu,et al.  Sentinel lymph node biopsy after neoadjuvant chemotherapy for breast cancer: retrospective comparative evaluation of clinically axillary lymph node positive and negative patients, including those with axillary lymph node metastases confirmed by fine needle aspiration , 2016, BMC Cancer.

[26]  Thomas George Olsen,et al.  Diagnostic Performance of Deep Learning Algorithms Applied to Three Common Diagnoses in Dermatopathology , 2018, Journal of pathology informatics.

[27]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[28]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[29]  H Svanholm,et al.  Prostatic carcinoma reproducibility of histologic grading. , 1985, Acta pathologica, microbiologica, et immunologica Scandinavica. Section A, Pathology.

[30]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[31]  Steven I Hajdu,et al.  A note from history: microscopic contributions of pioneer pathologists. , 2011, Annals of clinical and laboratory science.

[32]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[33]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[34]  D. Gleason,et al.  Histologic grading of prostate cancer: a perspective. , 1992, Human pathology.

[35]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[36]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[37]  Kaisa Liimatainen,et al.  Metastasis detection from whole slide images using local features and random forests , 2017, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Alexis B. Carter,et al.  Computational Pathology: A Path Ahead. , 2016, Archives of pathology & laboratory medicine.

[40]  Geraint Rees,et al.  Clinically applicable deep learning for diagnosis and referral in retinal disease , 2018, Nature Medicine.

[41]  Xavier Robin,et al.  pROC: an open-source package for R and S+ to analyze and compare ROC curves , 2011, BMC Bioinformatics.