A New Convolutional Neural Network Architecture for Automatic Segmentation of Overlapping Human Chromosomes

In clinical diagnosis, karyotyping is carried out to detect genetic disorders due to chromosomal aberrations. Accurate segmentation is crucial in this process that is mostly operated by experts. However, it is time-consuming and labor-intense to segment chromosomes and their overlapping regions. In this research, we look into the automatic segmentation of overlapping pairs of chromosomes. Different from standard semantic segmentation applications that mostly detect object regions or boundaries, this study attempts to predict not only non-overlapping regions but also the order of superposition and opaque regions of the underlying chromosomes. We propose a novel convolutional neural network called Compact Seg-UNet with enhanced deep feature learning capability and training efficacy. To address the issue of unrealistic images in use characterized by overlapping regions of higher color intensities, we propose a novel method to generate more realistic images with opaque overlapping regions. On the segmentation performance of overlapping chromosomes for this new dataset, our Compact Seg-UNet model achieves an average IOU score of 93.44% ± 0.26 which is significantly higher than the result of a simplified U-Net reported by literature by around 6.08%. The corresponding F1 score also increases from $$0.9262\pm 0.1188$$ to $$0.9596 \pm 0.0814$$ .