Probability Map Guided Bi-directional Recurrent UNet for Pancreas Segmentation

Pancreas segmentation in medical imaging data is of great significance for clinical pancreas diagnostics and treatment. However, the large population variations in the pancreas shape and volume cause enormous segmentation difficulties, even for state-of-the-art algorithms utilizing fully-convolutional neural networks (FCNs). Specifically, pancreas segmentation suffers from the loss of spatial information in 2D methods, and the high computational cost of 3D methods. To alleviate these problems, we propose a probabilistic-map-guided bi-directional recurrent UNet (PBR-UNet) architecture, which fuses intra-slice information and inter-slice probabilistic maps into a local 3D hybrid regularization scheme, which is followed by bi-directional recurrent network optimization. The PBR-UNet method consists of an initial estimation module for efficiently extracting pixel-level probabilistic maps and a primary segmentation module for propagating hybrid information through a 2.5D U-Net architecture. Specifically, local 3D information is inferred by combining an input image with the probabilistic maps of the adjacent slices into multichannel hybrid data, and then hierarchically aggregating the hybrid information of the entire segmentation network. Besides, a bi-directional recurrent optimization mechanism is developed to update the hybrid information in both the forward and the backward directions. This allows the proposed network to make full and optimal use of the local context information. Quantitative and qualitative evaluation was performed on the NIH Pancreas-CT dataset, and our proposed PBR-UNet method achieved better segmentation results with less computational cost compared to other state-of-the-art methods.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[3]  Changming Sun,et al.  RA-UNet: A Hybrid Deep Attention-Aware Network to Extract Liver and Tumor in CT Scans , 2018, Frontiers in Bioengineering and Biotechnology.

[4]  Ronald M. Summers,et al.  Spatial aggregation of holistically‐nested convolutional neural networks for automated pancreas localization and segmentation☆ , 2017, Medical Image Anal..

[5]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[6]  Matthew Lai,et al.  Deep Learning for Medical Image Segmentation , 2015, Deep Learning Applications in Medical Imaging.

[7]  Dong Yang,et al.  3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training , 2018, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[8]  Yuichiro Hayashi,et al.  An application of cascaded 3 D fully convolutional networks for medical image segmentation , 2018 .

[9]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[10]  Alan L. Yuille,et al.  A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation , 2017, 2018 International Conference on 3D Vision (3DV).

[11]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[12]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[13]  Yuichiro Hayashi,et al.  An application of cascaded 3D fully convolutional networks for medical image segmentation , 2018, Comput. Medical Imaging Graph..

[14]  Seyed-Ahmad Ahmadi,et al.  Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields , 2016, MICCAI.

[15]  Yan Wang,et al.  A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans , 2016, MICCAI.

[16]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Chengwen Chu,et al.  Multi‐atlas pancreas segmentation: Atlas selection based on vessel structure , 2017, Medical Image Anal..

[18]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[19]  Xinqi Gong,et al.  Hierarchical combinatorial deep learning architecture for pancreas segmentation of medical computed tomography cancer images , 2018, BMC Syst. Biol..

[20]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Milan Sonka,et al.  Deep LOGISMOS: Deep learning graph-based 3D segmentation of pancreatic tumors on CT scans , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[22]  Le Lu,et al.  Pancreas Segmentation in CT and MRI Images via Domain Specific Network Designing and Recurrent Neural Contextual Learning , 2018, ArXiv.

[23]  Alan L. Yuille,et al.  Recurrent Saliency Transformation Network: Incorporating Multi-stage Visual Cues for Small Organ Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[25]  Min Tang,et al.  Segmentation-by-detection: A cascade network for volumetric medical image segmentation , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[26]  Allan Hanbury,et al.  An Efficient Algorithm for Calculating the Exact Hausdorff Distance , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Sebastian J. Schlecht,et al.  Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks , 2017, ArXiv.

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Xin Yang,et al.  Towards Automated Semantic Segmentation in Prenatal Volumetric Ultrasound , 2019, IEEE Transactions on Medical Imaging.

[30]  Ronald M. Summers,et al.  Deep convolutional networks for pancreas segmentation in CT imaging , 2015, Medical Imaging.

[31]  Ronald M. Summers,et al.  A Bottom-Up Approach for Pancreas Segmentation Using Cascaded Superpixels and (Deep) Image Patch Labeling , 2015, IEEE Transactions on Image Processing.

[32]  Daguang Xu,et al.  3D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumes , 2017, MICCAI.

[33]  Daniel Rueckert,et al.  Regression Forest-Based Atlas Localization and Direction Specific Atlas Generation for Pancreas Segmentation , 2016, MICCAI.

[34]  Raúl San José Estépar,et al.  Multi-structure Segmentation from Partially Labeled Datasets. Application to Body Composition Measurements on CT Scans , 2018, RAMBO+BIA+TIA@MICCAI.

[35]  Fei Wu,et al.  Deep Q Learning Driven CT Pancreas Segmentation With Geometry-Aware U-Net , 2019, IEEE Transactions on Medical Imaging.

[36]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[37]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[38]  Chengwen Chu,et al.  Multi-organ Segmentation Based on Spatially-Divided Probabilistic Atlas from 3D Abdominal CT Images , 2013, MICCAI.

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Le Lu,et al.  Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function , 2017, ArXiv.

[41]  Barbara Villarini,et al.  Morphological and multi-level geometrical descriptor analysis in CT and MRI volumes for automatic pancreas segmentation , 2019, Comput. Medical Imaging Graph..

[42]  Xiao Xiang Zhu,et al.  Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multi-label aerial image classification , 2018, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[43]  Daniel Rueckert,et al.  Automated Abdominal Multi-Organ Segmentation With Subject-Specific Atlas Generation , 2013, IEEE Transactions on Medical Imaging.