An experimental study on the automatic segmentation of in situ hybridization-derived images

Histology is a well established branch of biology focused on the study of the microscopic anatomy of cells and tissues. Despite its importance, it is surprising to check that most literature devoted to histological image processing and analysis is focused on registration and 3D reconstruction of the whole brain. Therefore, there is a lack of research about automatic segmentation of anatomical brain structures in histological images. To bridge this gap, this paper introduces a comparative study of different segmentation techniques applied to this kind of images. A wide and representative image set has been collected to run experiments on the hippocampus, due to the importance of this anatomical district in learning, memory and spatial navigation. Seven approaches, from deterministic to non-deterministic ones, and from recent trends to classical computer vision techniques, have been compared using different standard metrics (Dice Similariry Coefficient, Jaccard Index, Hausdorff Distance, True Positive Rate and False Positive Rate). Proper statistical tests have been performed to draw accurate conclusions about the results. The best performance on this particular problem was obtained by a combination of Active Shape Models (optimized using Differential Evolution) with a refinement step based on Random Forests. This approach achieved an average Dice Similarity Coefficient of 0.89 with a standard deviation of 0.03.