Automatic segmentation of overlapping nuclei with high background variation using robust estimation and flexible contour models

We present a segmentation method that works for overlapping and closely packed nuclei in noisy images that have high variation in background intensity. The method has been tested on fluorescence in-situ hybridisation images of interphase leucocyte nuclei. Accurate segmentation is required in support of an automatic procedure for assaying telomere content on a per area per nucleus basis. The method first finds a single seed point for each nucleus that uniquely identifies that nucleus. Seed points are located by an efficient iterative mode-finding algorithm based on robust nonparametric density estimation. Acting simultaneously on all nuclei in the image, and using the seed points as origins, flexible closed contours are dilated until each nucleus is circumscribed. Unlike previous approaches, the contour equations include a repulsive term that prevents different contours from intersecting, thereby preserving the identity of nearby or overlapping nuclei, and the contour is adaptively remeshed for greater efficiency The locations of the seed points are not critical in providing an accurate segmentation. The advantage of this method from an implementation point of view is that the computation of seed points and contours is highly efficient and robust compared with alternative approaches. The method is illustrated using data from a clinical pilot study.