U-Next: A Novel Convolution Neural Network With an Aggregation U-Net Architecture for Gallstone Segmentation in CT Images

The incidence of cholelithiasis is more than 10% in the natural population. It is a common and frequently occurring disease worldwide. The lesions are predominantly sand-like and are difficult to distinguish in medical images. Although ultrasound is often the first-line technique for diagnosing cholelithiasis, CT plays an important role in diagnosing complications related to gallstones. To the best of our knowledge, no effective method has been proposed for segmenting CT images of gallstones. To address this difficulty, we have proposed a novel deep learning segmentation model, U-NeXt, which can automatically provide the importance of different shapes and sizes of target structures in images. a) We proposed attention up-sampling blocks for up-sampling. b) We also proposed a spatial pyramid pooling of skip connections, which is called SkipSPP. A series of convolution layers are connected by jumping connections to generate multi-scale features. c) We employed a dense connection to many details of the model to extract more features. In addition, we contributed a new data set for gallstone segmentation, which is co-labelled by three chief physicians from the hepatobiliary department and the medical imaging department of Shandong Province Third Hospital, China. The data set included 5,350 images from 726 patients. Compared with the previous best CT image segmentation method for gallstones, the intersection over union (IoU) improved by 24%, and compared with the medical image segmentation baseline model, the IoU improved by 7%, with a satisfactory segmentation effect. We extended U-NeXt to the nuclei segmentation task, showing that the model has good generalizability.

[1]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[2]  Hao Chen,et al.  Automatic 3D Cardiovascular MR Segmentation with Densely-Connected Volumetric ConvNets , 2017, MICCAI.

[3]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[4]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[5]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Trevor Darrell,et al.  Deep Layer Aggregation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Tat-Seng Chua,et al.  SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Wang Hong-yan,et al.  A lore-based FCM algorithm for gallstone CT images segmentation , 2015, 2015 10th International Conference on Computer Science & Education (ICCSE).

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  Hao Chen,et al.  Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images , 2017, AAAI.

[13]  Vineet Khandelwal,et al.  Automated segmentation of gallstones in ultrasound images , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[14]  Zhen Yang,et al.  Automatic gallbladder and gallstone regions segmentation in ultrasound image , 2017, International Journal of Computer Assisted Radiology and Surgery.

[15]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[16]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Weiying Xie,et al.  Gallstone segmentation and extraction from ultrasound images using level set model , 2013, 2013 ISSNIP Biosignals and Biorobotics Conference: Biosignals and Robotics for Better and Safer Living (BRC).

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[23]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[24]  Alexander J. Smola,et al.  Stacked Attention Networks for Image Question Answering , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Kate Saenko,et al.  Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering , 2015, ECCV.

[26]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[27]  Zhiqiang Shen,et al.  DSOD: Learning Deeply Supervised Object Detectors from Scratch , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).