Cell‐type based semantic segmentation of histopathological images using deep convolutional neural networks

Histopathological whole‐slide image (WSI) analysis is still one of the most important ways to identify regions of cancer risk. For cancer in which early diagnosis is vital, pathologists are at the center of the decision‐making process. Thanks to the widespread use of digital pathology and the development of artificial intelligence methods, automatic histopathological image analysis methods help pathologists in their decision‐making process. In this process, rather than producing labels for whole‐slide image patches, semantic segmentation is very useful, which facilitates the pathologists’ interpretation. In this study, automatic semantic segmentation based on cell type is proposed for the first time in the literature using novel deep convolutional networks structure (DCNN). We presents semantic information on four classes, including white areas in the whole‐slide image, tissue without cells, tissue with normal cells and tissue with cancerous cells. This visual information presented to the pathologist is an easy‐to‐understand picture of the status of the cells and their implications for the spread of cancerous cells. A new DCNN architecture is created, inspired by the residual network and deconvolution network architecture. Our network is trained end‐to‐end manner with histopathological image patches for cell structures to be more discriminative. The proposed method not only produces more successful results than other state‐of‐art semantic segmentation algorithms with 9.2% training error and 88.89% F‐score for test, but also has the most important advantage in that it has the ability to generate automatic information about the cancer and also provides information that pathologists can quickly interpret.

[1]  Jianzhong Wu,et al.  Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images , 2016, IEEE Transactions on Medical Imaging.

[2]  Aimin Hao,et al.  An Extended Type Cell Detection and Counting Method based on FCN , 2017, 2017 IEEE 17th International Conference on Bioinformatics and Bioengineering (BIBE).

[3]  Zhiguo Jiang,et al.  Feature extraction from histopathological images based on nucleus-guided convolutional neural network for breast lesion classification , 2017, Pattern Recognit..

[4]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xin Zhao,et al.  Locality-Sensitive Deconvolution Networks with Gated Fusion for RGB-D Indoor Semantic Segmentation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  S. Fedewa,et al.  Cancer screening in the United States, 2017: A review of current American Cancer Society guidelines and current issues in cancer screening , 2017, CA: a cancer journal for clinicians.

[7]  Zhuowen Tu,et al.  Weakly supervised histopathology cancer image segmentation and classification , 2014, Medical Image Anal..

[8]  Jun Xu,et al.  Sparse Non-negative Matrix Factorization (SNMF) based color unmixing for breast histopathological image analysis , 2015, Comput. Medical Imaging Graph..

[9]  Shihui Ying,et al.  Histopathological image classification with bilinear convolutional neural networks , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[10]  Shengcai Liao,et al.  Multi-label convolutional neural network based pedestrian attribute classification , 2017, Image Vis. Comput..

[11]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Rebecca L. Siegel Mph,et al.  Cancer statistics, 2018 , 2018 .

[13]  Zhipeng Jia,et al.  Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features , 2017, BMC Bioinformatics.

[14]  Ruba Soundar Kathavarayan,et al.  Segmentation of hepatocellular carcinoma and dysplastic liver tumors in histopathology images using area based adaptive expectation maximization , 2016, Multimedia Tools and Applications.

[15]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[16]  Hiroshi Fujita,et al.  Automated Classification of Lung Cancer Types from Cytological Images Using Deep Convolutional Neural Networks , 2017, BioMed research international.

[17]  A. Madabhushi,et al.  Gland Segmentation and Computerized Gleason Grading of Prostate Histology by Integrating Low-, High-level and Domain Specific Information , 2007 .

[18]  Shadi Albarqouni,et al.  AggNet : Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images , 2016 .

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Anabik Pal,et al.  Psoriasis skin biopsy image segmentation using Deep Convolutional Neural Network , 2018, Comput. Methods Programs Biomed..

[21]  Zhiguo Jiang,et al.  Histopathological Whole Slide Image Analysis Using Context-Based CBIR , 2018, IEEE Transactions on Medical Imaging.

[22]  Yousef Al-Kofahi,et al.  Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images , 2010, IEEE Transactions on Biomedical Engineering.

[23]  Lin Yang,et al.  Robust Segmentation of Overlapping Cells in Histopathology Specimens Using Parallel Seed Detection and Repulsive Level Set , 2012, IEEE Transactions on Biomedical Engineering.

[24]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[25]  Zhipeng Jia,et al.  Constrained Deep Weak Supervision for Histopathology Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[26]  Nassir Navab,et al.  AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[27]  Fazly Salleh Abas,et al.  Tumor detection and whole slide classification of H&E lymph node images using convolutional neural network , 2017, 2017 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[28]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Luiz Eduardo Soares de Oliveira,et al.  Deep features for breast cancer histopathological image classification , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[30]  Hai Su,et al.  Robust Cell Detection of Histopathological Brain Tumor Images Using Sparse Reconstruction and Adaptive Dictionary Selection , 2016, IEEE Transactions on Medical Imaging.

[31]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[32]  Shuiwang Ji,et al.  Residual Deconvolutional Networks for Brain Electron Microscopy Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[33]  Nico Karssemeijer,et al.  Automated Detection of DCIS in Whole-Slide H&E Stained Breast Histopathology Images , 2016, IEEE Transactions on Medical Imaging.

[34]  Linda G. Shapiro,et al.  Multi-Instance Multi-Label Learning for Multi-Class Classification of Whole Slide Breast Histopathology Images , 2018, IEEE Transactions on Medical Imaging.

[35]  Nico Karssemeijer,et al.  Stain Specific Standardization of Whole-Slide Histopathological Images , 2016, IEEE Transactions on Medical Imaging.

[36]  Hai Su,et al.  High-throughput histopathological image analysis via robust cell segmentation and hashing , 2015, Medical Image Anal..

[37]  Xavier Farré,et al.  The untapped potential of digital pathology in prostate cancer diagnosis and medical education in sub-Saharan Africa , 2018 .

[38]  Chandan Chakraborty,et al.  Her2Net: A Deep Framework for Semantic Segmentation and Classification of Cell Membranes and Nuclei in Breast Cancer Evaluation , 2018, IEEE Transactions on Image Processing.

[39]  Yunchao Wei,et al.  STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  A. Jemal,et al.  Cancer statistics, 2018 , 2018, CA: a cancer journal for clinicians.

[41]  Adel Hafiane,et al.  Integrating segmentation with deep learning for enhanced classification of epithelial and stromal tissues in H&E images , 2017, Pattern Recognit. Lett..

[42]  Borivoj Vojnovic,et al.  An image analysis‐based approach for automated counting of cancer cell nuclei in tissue sections , 2003, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[43]  Mrinal Mandal,et al.  A robust automatic nuclei segmentation technique for quantitative histopathological image analysis. , 2012, Analytical and quantitative cytopathology and histopathology.

[44]  Constantine Katsinis,et al.  Large-scale computations on histology images reveal grade-differentiating parameters for breast cancer , 2006, BMC Medical Imaging.

[45]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[46]  Shihui Ying,et al.  Histopathological Image Classification With Color Pattern Random Binary Hashing-Based PCANet and Matrix-Form Classifier , 2017, IEEE Journal of Biomedical and Health Informatics.

[47]  Johan Isaksson,et al.  Semantic segmentation of microscopic images of H&E stained prostatic tissue using CNN , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).