Cytokeratin-Supervised Deep Learning for Automatic Recognition of Epithelial Cells in Breast Cancers Stained for ER, PR, and Ki-67

Immunohistochemistry (IHC) of ER, PR, and Ki-67 are routinely used assays in breast cancer diagnostics. Determination of the proportion of stained cells (labeling index) should be restricted on malignant epithelial cells, carefully avoiding tumor infiltrating stroma and inflammatory cells. Here, we developed a deep learning based digital mask for automated epithelial cell detection using fluoro-chromogenic cytokeratin-Ki-67 double staining and sequential hematoxylin-IHC staining as training material. A partially pre-trained deep convolutional neural network was fine-tuned using image batches from 152 patient samples of invasive breast tumors. Validity of the trained digital epithelial cell masks was studied with 366 images captured from 98 unseen samples, by comparing the epithelial cell masks to cytokeratin images and by visual evaluation of the brightfield images performed by two pathologists. A good discrimination of epithelial cells was achieved (AUC of mean ROC = 0.93; defined as the area under mean receiver operating characteristics), and well in concordance with pathologists’ visual assessment (4.01/5 and 4.67/5). The effect of epithelial cell masking on the Ki-67 labeling index was substantial. 52 tumor images initially classified as low proliferation (Ki-67 < 14%) without epithelial cell masking were re-classified as high proliferation (Ki-67 ≥ 14%) after applying the deep learning based epithelial cell mask. The digital epithelial cell masks were found applicable also to IHC of ER and PR. We conclude that deep learning can be applied to detect carcinoma cells in breast cancer samples stained with conventional brightfield IHC.

[1]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[2]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[7]  Ewert Bengtsson,et al.  Image Analysis in Digital Pathology: Combining Automated Assessment of Ki67 Staining Quality with Calculation of Ki67 Cell Proliferation Index , 2018, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[8]  David L Rimm,et al.  Ki67 reproducibility using digital image analysis: an inter-platform and inter-operator study , 2018, Laboratory Investigation.

[9]  A. Ruifrok,et al.  Quantification of histochemical staining by color deconvolution. , 2001, Analytical and quantitative cytology and histology.

[10]  Teemu Tolonen,et al.  Optimized JPEG 2000 Compression for Efficient Storage of Histopathological Whole-Slide Images , 2018, Journal of pathology informatics.

[11]  Kaisa Liimatainen,et al.  Dual Structured Convolutional Neural Network with Feature Augmentation for Quantitative Characterization of Tissue Histology , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[12]  Rikke Riber-Hansen,et al.  Proliferation assessment in breast carcinomas using digital image analysis based on virtual Ki67/cytokeratin double staining , 2016, Breast Cancer Research and Treatment.

[13]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[14]  Bert van der Vegt,et al.  Digital image analysis of Ki67 proliferation index in breast cancer using virtual dual staining on whole tissue sections: clinical validation and inter-platform agreement , 2018, Breast Cancer Research and Treatment.

[15]  Karl Rohr,et al.  Predicting breast tumor proliferation from whole‐slide images: The TUPAC16 challenge , 2018, Medical Image Anal..

[16]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[17]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[18]  Linda G. Shapiro,et al.  Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks , 2018, Pattern Recognit..

[19]  Kevin W Eliceiri,et al.  NIH Image to ImageJ: 25 years of image analysis , 2012, Nature Methods.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  S. Barni,et al.  Prognostic value of different cut-off levels of Ki-67 in breast cancer: a systematic review and meta-analysis of 64,196 patients , 2015, Breast Cancer Research and Treatment.

[22]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[23]  Jack Cuzick,et al.  Assessment of Ki67 in breast cancer: recommendations from the International Ki67 in Breast Cancer working group. , 2011, Journal of the National Cancer Institute.

[24]  R. Gelber,et al.  Strategies for subtypes—dealing with the diversity of breast cancer: highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2011 , 2011, Annals of oncology : official journal of the European Society for Medical Oncology.

[25]  Vilppu J Tuominen,et al.  ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67 , 2010, Breast Cancer Research.

[26]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[27]  Marc Aubreville,et al.  Field of Interest Prediction for Computer-Aided Mitotic Count , 2019, ArXiv.