Deep Voting and Structured Regression for Microscopy Image Analysis

Abstract Robust and accurate nuclei localization in microscopy images can provide crucial clues for accurate computer-aided diagnosis. In this chapter, we present two methods that rely on convolutional neural networks (CNNs) to solve this problem. The first one is named as deep voting, which is a CNN based hough voting method used to localize nucleus centroids that exhibit heavy cluttering and morphological variations. It mainly consists of the following two parts: (i) Given an input image, this model maps every local testing image patch to the proposed target information, which consists of several pairs of voting offset vectors and voting confidence. Voting offset vectors are used to specify the pixel coordinates each local testing image patch votes to, and the corresponding voting confidence is used as weight assigned to each vote. (ii) We collect the weighted votes from all the testing image patches and compute the final voting density map for the entire testing image in a way similar to Parzen-window estimation. The final nucleus positions are then identified by searching the local maxima of the density map. The second one is a novel CNN based structured regression model, which is shown to be able to handle touching cells, inhomogeneous background noises, and large variations in sizes and shapes. Given an input image patch, instead of providing a single class label like many traditional methods, it will generate the structured outputs (referred to as proximity patches). These proximity patches, which exhibit higher values for pixels near cell centers, will then be gathered from all testing image patches and fused to obtain the final proximity map, where the maximum positions indicate the cell centroids. Both methods only require a few training images with weak annotations (just one click near the center of the object). Experimental results demonstrate that the proposed method achieves significantly improved performance compared to the state-of-the-art methods and shows strong robustness when dealing with the complex touching cells, weak staining, and fuzzy boundaries.