A unified deep-learning network to accurately segment insulin granules of different animal models imaged under different electron microscopy methodologies

Insulin is important for body metabolism regulation and glucose homeostasis, and its dysregulation often leads to metabolic syndrome (MS) and diabetes. Insulin is normally stored in large dense-core vesicles (LDCVs) in pancreatic beta cells, and significant reductions in the number, size, gray level and density of insulin granules confer diabetes both in mice (Xue et al., 2012) and humans (Masini et al., 2012). Due to the difficulty of obtaining human islet samples, many works use mice as the animal model. However, the architecture of normal islets in humans differs significantly from that of rodents (Cabrera et al., 2006). Beta cells in the mouse islet core are surrounded by the mantle comprising of alpha and delta cells, whereas alpha, beta and delta cells are intermingled in human islets. The structural differences suggest a possible difference in islet function. In this sense, non-human primates such as rhesus monkeys are a better model, as their islets share a similar architecture with humans (Cabrera et al., 2006). The quantitative nature of the insulin granules within monkey islet beta cells, and whether they change during metabolic dysregulation remain to be explored. Under the electron microscope (EM), insulin granules are usually spherical organelles containing an electron dense-core separated from the surrounding membrane by a halo, with a size of ranged from 100–800 nm (MacDonald et al., 2006) in mouse beta cells. This number is estimated as ∼10,000 per beta cell. Because thin-section EM do not necessarily provide the correct spatial coordination of granules within one beta cell, recent years have witnessed the emergence of volumetric electron microscopy techniques such as electron tomography and focused ion beam scanning electron microscopy (FIB-SEM) (Briggman and Bock, 2012). For the first time, we collected three-dimensional images of pancreatic beta cells in wild type (WT) and MS rhesus monkeys with a FIB-SEM and manually annotated granules from a relatively small number of images. Because the morphological and structural natures of insulin granules are important for their optimal function, quantitative and automatic analysis of insulin granules in islets is important. Manually segmenting densely distributed LDCVs is a laborintensive task due to the big datasets brought by saturated and continuous sampling in the lateral and axial axes. Although several semi-automated segmentation methods for rodent LDCVs have been proposed (Diaz et al., 2010), they are built on time-consuming and human-designed features that cannot adapt to micrographs of different magnification and are extremely prone to errors for images with low signalto-noise ratios. Nevertheless, the machine learning field has witnessed a flourishing of “deep-learning” algorithms. Since AlexNet outperformed all other algorithms by a large margin in the ImageNet contest in 2012, a variety of deep-learning methods for image segmentation have been widely used, including the standard convolutional networks (CNN) (Van Valen et al., 2016), and fully convolutional networks (FCN) (Long, 2014). Recently, multi-scale features, dilated convolutions, context encoding, conditional random fields (CRFs) are incorporated to FCNs to improve spatial resolution, bringing more novel and complicated network structures such as Tiramisu (Jegou et al., 2017), Deeplab (Chen et al., 2018). However, different from natural images, we only have a small electron micrograph dataset annotated, in which insulin granules only occupy a small portion of the image. In order to prevent overfitting, we take concise deep-learning networks as the starting point, such as U-Net (Ronneberger et al., 2015). To automatically and precisely segment the insulin granules with different sizes and pattern features in EM images, we built and trained a multi-branch fully convolutional network (MFCN), which consists of three modules: a multi-scale inception module, a multi-branch sampling module, and a multi-scale ensemble module (Fig. 1). The “multi-scale” design was inspired by the naïve inception module (Szegedy et al., 2015), which uses different sizes kernels to extract both coarse and fine grained features. Although some papers have proposed that a stack of two 3 × 3 convolutional