Point-Supervised Segmentation Of Microscopy Images And Volumes Via Objectness Regularization

Annotation is a major hurdle in the semantic segmentation of microscopy images and volumes due to its prerequisite expertise and effort. This work enables the training of semantic segmentation networks on images with only a single point for training per instance, an extreme case of weak supervision which drastically reduces the burden of annotation. Our approach has two key aspects: (1) we construct a graph-theoretic soft-segmentation using individual seeds to be used within a regularizer during training and (2) we use an objective function that enables learning from the constructed soft-labels. We achieve competitive results against the state-of-the-art in point-supervised semantic segmentation on challenging datasets in digital pathology. Finally, we scale our methodology to point-supervised segmentation in 3D fluorescence microscopy volumes, obviating the need for arduous manual volumetric delineation. Our code is freely available.

[1]  Kristen Grauman,et al.  Pixel Objectness: Learning to Segment Generic Objects Automatically in Images and Videos , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Danielle B. Gutierrez,et al.  Quantitative autofluorescence and cell density maps of the human retinal pigment epithelium. , 2014, Investigative ophthalmology & visual science.

[3]  Dimitris N. Metaxas,et al.  Weakly Supervised Deep Nuclei Segmentation using Points Annotation in Histopathology Images , 2019, MIDL.

[4]  Ismail Ben Ayed,et al.  On Regularized Losses for Weakly-supervised CNN Segmentation , 2018, ECCV.

[5]  Jin Tae Kwak,et al.  Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images , 2018, Medical Image Anal..

[6]  Inwan Yoo,et al.  PseudoEdgeNet: Nuclei Segmentation only with Point Annotations , 2019, MICCAI.

[7]  Nassir Navab,et al.  Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images , 2016, IEEE Transactions on Medical Imaging.

[8]  Anne E Carpenter,et al.  Annotated high-throughput microscopy image sets for validation , 2012, Nature Methods.

[9]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[10]  C. Curcio,et al.  Lipofuscin redistribution and loss accompanied by cytoskeletal stress in retinal pigment epithelium of eyes with age-related macular degeneration. , 2015, Investigative ophthalmology & visual science.

[11]  Nir Shavit,et al.  Deep Learning is Robust to Massive Label Noise , 2017, ArXiv.

[12]  Alexander Schrijver,et al.  On the history of the transportation and maximum flow problems , 2002, Math. Program..

[13]  Luc Van Gool,et al.  Deep Extreme Cut: From Extreme Points to Object Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Mark W. Schmidt,et al.  Where are the Blobs: Counting by Localization with Point Supervision , 2018, ECCV.

[16]  C. Curcio,et al.  Autofluorescent Granules of the Human Retinal Pigment Epithelium: Phenotypes, Intracellular Distribution, and Age-Related Topography , 2020, Investigative ophthalmology & visual science.

[17]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Yuri Boykov,et al.  Normalized Cut Loss for Weakly-Supervised CNN Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Horst K. Hahn,et al.  Deep learning nuclei detection: A simple approach can deliver state-of-the-art results , 2018, Comput. Medical Imaging Graph..

[20]  Bernt Schiele,et al.  Simple Does It: Weakly Supervised Instance and Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).