Deep Learning for Whole Slide Image Analysis: An Overview

The widespread adoption of whole slide imaging has increased the demand for effective and efficient gigapixel image analysis. Deep learning is at the forefront of computer vision, showcasing significant improvements over previous methodologies on visual understanding. However, whole slide images have billions of pixels and suffer from high morphological heterogeneity as well as from different types of artifacts. Collectively, these impede the conventional use of deep learning. For the clinical translation of deep learning solutions to become a reality, these challenges need to be addressed. In this paper, we review work on the interdisciplinary attempt of training deep neural networks using whole slide images, and highlight the different ideas underlying these methodologies.

[1]  Neofytos Dimitriou,et al.  A principled machine learning framework improves accuracy of stage II colorectal cancer prognosis , 2018, npj Digital Medicine.

[2]  Ognjen Arandjelovic,et al.  Colorectal Cancer Outcome Prediction from H&E Whole Slide Images using Machine Learning and Automatically Inferred Phenotype Profiles , 2019, ArXiv.

[3]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Nasir M. Rajpoot,et al.  Learning Where to See: A Novel Attention Model for Automated Immunohistochemical Scoring , 2019, IEEE Transactions on Medical Imaging.

[5]  Yi Li,et al.  Cancer Metastasis Detection With Neural Conditional Random Field , 2018, ArXiv.

[6]  David A. Clunie,et al.  Dual-Personality DICOM-TIFF for Whole Slide Images: A Migration Technique for Legacy Software , 2019, Journal of pathology informatics.

[7]  P. Caie,et al.  Novel histopathologic feature identified through image analysis augments stage II colorectal cancer clinical reporting , 2016, Oncotarget.

[8]  M. Salto‐Tellez,et al.  Digital pathology and image analysis in tissue biomarker research. , 2014, Methods.

[9]  Ron Kikinis,et al.  Implementing the DICOM Standard for Digital Pathology , 2018, Journal of pathology informatics.

[10]  Yee-Wah Tsang,et al.  Validation of digital pathology imaging for primary histopathological diagnosis , 2016, Histopathology.

[11]  Alexis B. Carter,et al.  Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. , 2013, Archives of pathology & laboratory medicine.

[12]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[14]  Olivier Gevaert,et al.  Deep Recurrent Attention Models for Histopathological Image Analysis , 2018, bioRxiv.

[15]  Francesco Ciompi,et al.  Neural Image Compression for Gigapixel Histopathology Image Analysis , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[17]  Charles E Kahn,et al.  DICOM and radiology: past, present, and future. , 2007, Journal of the American College of Radiology : JACR.

[18]  Nicolas Brieu,et al.  Automated tumour budding quantification by machine learning augments TNM staging in muscle-invasive bladder cancer prognosis , 2019, Scientific Reports.

[19]  Nathalie Harder,et al.  Automatic discovery of image-based signatures for ipilimumab response prediction in malignant melanoma , 2019, Scientific Reports.

[20]  Derek R. Magee,et al.  Colour Normalisation in Digital Histopathology Images , 2009 .

[21]  Catarina Eloy,et al.  BACH: Grand Challenge on Breast Cancer Histology Images , 2018, Medical Image Anal..

[22]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[23]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[24]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[25]  Nikolaos Doulamis,et al.  Deep Learning for Computer Vision: A Brief Review , 2018, Comput. Intell. Neurosci..

[26]  Eric W. Tramel,et al.  Classification and Disease Localization in Histopathology Using Only Global Labels: A Weakly-Supervised Approach , 2018, ArXiv.

[27]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[28]  Daniel Zelterman,et al.  Multiplex Quantitative Analysis of Tumor-Infiltrating Lymphocytes and Immunotherapy Outcome in Metastatic Melanoma , 2019, Clinical Cancer Research.

[29]  Ghassan Hamarneh,et al.  Predicting Cancer with a Recurrent Visual Attention Model for Histopathology Images , 2018, MICCAI.

[30]  Max Welling,et al.  Attention-based Deep Multiple Instance Learning , 2018, ICML.

[31]  Tae-Yeong Kwak,et al.  Artificial Intelligence in Pathology , 2018, Journal of pathology and translational medicine.

[32]  Xin Wang,et al.  Cancer Metastasis Detection via Spatially Structured Deep Network , 2017, IPMI.

[33]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[34]  Shaoqun Zeng,et al.  From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge , 2019, IEEE Transactions on Medical Imaging.

[35]  Cigdem Gunduz-Demir,et al.  Unsupervised Feature Extraction via Deep Learning for Histopathological Classification of Colon Tissue Images , 2019, IEEE Transactions on Medical Imaging.

[36]  Thomas J. Fuchs,et al.  Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in Pathology , 2018, ArXiv.

[37]  Nicolas Brieu,et al.  Context-based interpolation of coarse deep learning prediction maps for the segmentation of fine structures in immunofluorescence images , 2018, Medical Imaging.

[38]  Andrew H. Beck,et al.  Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival , 2011, Science Translational Medicine.

[39]  Karl Rohr,et al.  Predicting breast tumor proliferation from whole‐slide images: The TUPAC16 challenge , 2018, Medical Image Anal..

[40]  K. Sirinukunwattana,et al.  Image-based consensus molecular subtype (imCMS) classification of colorectal cancer using deep learning , 2020, Gut.

[41]  Jens Rittscher,et al.  Image-based consensus molecular subtype classification (imCMS) of colorectal cancer using deep learning , 2019, bioRxiv.

[42]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[43]  Junzhou Huang,et al.  WSISA: Making Survival Prediction from Whole Slide Histopathological Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Kirk E. Smith,et al.  Reengineering Workflow for Curation of DICOM Datasets , 2018, Journal of Digital Imaging.

[45]  Peter H. N. de With,et al.  Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces , 2018, Medical Imaging.

[46]  Ognjen Arandjelovic,et al.  Ancient Roman Coin Recognition in the Wild Using Deep Learning Based Recognition of Artistically Depicted Face Profiles , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[47]  Verónica Vilaplana,et al.  Monte-Carlo Sampling Applied to Multiple Instance Learning for Histological Image Classification , 2018, DLMIA/ML-CDS@MICCAI.

[48]  Jeffrey A. Golden,et al.  Deep Learning Algorithms for Detection of Lymph Node Metastases From Breast Cancer: Helping Artificial Intelligence Be Seen. , 2017, JAMA.

[49]  Peter D Caie,et al.  Human tissue in systems medicine , 2013, The FEBS journal.

[50]  Ramakrishnan Mukundan,et al.  HER2 challenge contest: a detailed assessment of automated HER2 scoring algorithms in whole slide images of breast cancer tissues , 2017, Histopathology.

[51]  Peter D Caie,et al.  Automated Analysis of Lymphocytic Infiltration, Tumor Budding, and Their Spatial Relationship Improves Prognostic Accuracy in Colorectal Cancer , 2019, Cancer Immunology Research.

[52]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.