Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images

Nuclei segmentation is a fundamental task in histopathology image analysis. Typically, such segmentation tasks require significant effort to manually generate accurate pixel-wise annotations for fully supervised training. To alleviate such tedious and manual effort, in this paper we propose a novel weakly supervised segmentation framework based on partial points annotation, i.e., only a small portion of nuclei locations in each image are labeled. The framework consists of two learning stages. In the first stage, we design a semi-supervised strategy to learn a detection model from partially labeled nuclei locations. Specifically, an extended Gaussian mask is designed to train an initial model with partially labeled data. Then, self-training with background propagation is proposed to make use of the unlabeled regions to boost nuclei detection and suppress false positives. In the second stage, a segmentation model is trained from the detected nuclei locations in a weakly-supervised fashion. Two types of coarse labels with complementary information are derived from the detected points and are then utilized to train a deep neural network. The fully-connected conditional random field loss is utilized in training to further refine the model without introducing extra computational complexity during inference. The proposed method is extensively evaluated on two nuclei segmentation datasets. The experimental results demonstrate that our method can achieve competitive performance compared to the fully supervised counterpart and the state-of-the-art methods while requiring significantly less annotation effort.

[1]  Hai Su,et al.  Beyond Classification: Structured Regression for Robust Cell Detection Using Convolutional Neural Network , 2015, MICCAI.

[2]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[3]  Dimitris N. Metaxas,et al.  Joint Segmentation and Fine-Grained Classification of Nuclei in Histopathology Images , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[4]  A. Huisman,et al.  Automatic Nuclei Segmentation in H&E Stained Breast Cancer Histopathology Images , 2013, PloS one.

[5]  Hai Su,et al.  High-throughput histopathological image analysis via robust cell segmentation and hashing , 2015, Medical Image Anal..

[6]  Thomas Walter,et al.  Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map , 2019, IEEE Transactions on Medical Imaging.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Konstantinos Kamnitsas,et al.  DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[9]  Hai Su,et al.  Efficient and robust cell detection: A structured regression approach , 2018, Medical Image Anal..

[10]  Nasir M. Rajpoot,et al.  A Stochastic Polygons Model for Glandular Structures in Colon Histology Images , 2015, IEEE Transactions on Medical Imaging.

[11]  Hai Su,et al.  Robust Cell Detection and Segmentation in Histopathological Images Using Sparse Reconstruction and Stacked Denoising Autoencoders , 2015, MICCAI.

[12]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Dimitris N. Metaxas,et al.  Weakly Supervised Deep Nuclei Segmentation using Points Annotation in Histopathology Images , 2019, MIDL.

[15]  Ismail Ben Ayed,et al.  On Regularized Losses for Weakly-supervised CNN Segmentation , 2018, ECCV.

[16]  George Papandreou,et al.  Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Zhiqiang Hu,et al.  Signet Ring Cell Detection with a Semi-supervised Learning Framework , 2019, IPMI.

[18]  Hao Chen,et al.  SFCN-OPI: Detection and Fine-grained Classification of Nuclei Using Sibling FCN with Objectness Prior Interaction , 2017, AAAI.

[19]  Jian Sun,et al.  BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[20]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[21]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[22]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[23]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[24]  Wei Liu,et al.  Towards Large-Scale Histopathological Image Analysis: Hashing-Based Image Retrieval , 2015, IEEE Transactions on Medical Imaging.

[25]  Lin Yang,et al.  Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review , 2016, IEEE Reviews in Biomedical Engineering.

[26]  Andrew Adams,et al.  Fast High‐Dimensional Filtering Using the Permutohedral Lattice , 2010, Comput. Graph. Forum.

[27]  Zhiqiang Hu,et al.  Accurate Nuclear Segmentation with Center Vector Encoding , 2019, IPMI.

[28]  Thomas Walter,et al.  Nuclei segmentation in histopathology images using deep neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[29]  Lin Yang,et al.  Deep Learning Based Instance Segmentation in 3D Biomedical Images Using Weak Annotation , 2018, MICCAI.

[30]  Yousef Al-Kofahi,et al.  Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images , 2010, IEEE Transactions on Biomedical Engineering.

[31]  Jianzhong Wu,et al.  Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images , 2016, IEEE Transactions on Medical Imaging.

[32]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[33]  Hui Kong,et al.  Partitioning Histopathological Images: An Integrated Framework for Supervised Color-Texture Segmentation and Cell Splitting , 2011, IEEE Transactions on Medical Imaging.

[34]  Lin Yang,et al.  BoxNet: Deep Learning Based Biomedical Image Segmentation Using Boxes Only Annotation , 2018, ArXiv.

[35]  Trevor Darrell,et al.  Constrained Convolutional Neural Networks for Weakly Supervised Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Surabhi Bhargava,et al.  A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology , 2017, IEEE Transactions on Medical Imaging.

[37]  David B. A. Epstein,et al.  Micro‐Net: A unified model for segmentation of various objects in microscopy images , 2018, Medical Image Anal..

[38]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[39]  Dimitris N. Metaxas,et al.  Improving Nuclei/Gland Instance Segmentation in Histopathology Images by Full Resolution Neural Network and Spatial Constrained Loss , 2019, MICCAI.

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[42]  Joel H. Saltz,et al.  Robust Histopathology Image Analysis: To Label or to Synthesize? , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[44]  Lin Yang,et al.  An Automatic Learning-Based Framework for Robust Nucleus Segmentation , 2016, IEEE Transactions on Medical Imaging.

[45]  Dimitris N. Metaxas,et al.  Nuclei Segmentation Using Mixed Points and Masks Selected From Uncertainty , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[46]  Eric Granger,et al.  Constrained‐CNN losses for weakly supervised segmentation☆ , 2018, Medical Image Anal..

[47]  Richard J. Chen,et al.  Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images , 2018, IEEE Transactions on Medical Imaging.