Shape Mask Generator: Learning to Refine Shape Priors for Segmenting Overlapping Cervical Cytoplasms

Segmenting overlapping cytoplasm of cervical cells plays a crucial role in cervical cancer screening. This task, however, is rather challenging, mainly because intensity (or color) information in the overlapping region is deficient for recognizing occluded boundary parts. Existing methods attempt to compensate intensity deficiency by exploiting shape priors, but shape priors modeled by them have a weak representation ability, and hence their segmentation results are often visually implausible in shape. In this paper, we propose a conceptually simple and effective technique, called shape mask generator, for segmenting overlapping cytoplasms. The key idea is to progressively refine shape priors by learning so that they can accurately represent most cytoplasms’ shape. Specifically, we model shape priors from shape templates and feed them to the shape mask generator that generates a shape mask for the cytoplasm as the segmentation result. Shape priors are refined by minimizing the ‘generating residual’ in the training dataset, which is designed to have a smaller value when the shape mask generator producing shape masks that are more consistent with the image information. The introduced method is assessed on two datasets, and the empirical evidence shows that it is effective, outperforming existing methods.