Pancreas segmentation using a dual-input v-mesh network

Accurate segmentation of the pancreas from abdomen scans is crucial for the diagnosis and treatment of pancreatic diseases. However, the pancreas is a small, soft and elastic abdominal organ with high anatomical variability and has a low tissue contrast in computed tomography (CT) scans, which makes segmentation tasks challenging. To address this challenge, we propose a dual-input v-mesh fully convolutional network (FCN) to segment the pancreas in abdominal CT images. Specifically, dual inputs, i.e., original CT scans and images processed by a contrast-specific graph-based visual saliency (GBVS) algorithm, are simultaneously sent to the network to improve the contrast of the pancreas and other soft tissues. To further enhance the ability to learn context information and extract distinct features, a v-mesh FCN with an attention mechanism is initially utilized. In addition, we propose a spatial transformation and fusion (SF) module to better capture the geometric information of the pancreas and facilitate feature map fusion. We compare the performance of our method with several baseline and state-of-the-art methods on the publicly available NIH dataset. The comparison results show that our proposed dual-input v-mesh FCN model outperforms previous methods in terms of the Dice similarity coefficient (DSC), positive predictive value (PPV), sensitivity (SEN), average surface distance (ASD) and Hausdorff distance (HD). Moreover, ablation studies show that our proposed modules/structures are critical for effective pancreas segmentation.

[1]  Rabia Tahir,et al.  Improve Crowd Size Estimation by Leveraging Deformable Convolutional Neural Network and Deformable Region of Interest , 2019, 2019 6th International Conference on Systems and Informatics (ICSAI).

[2]  Ronald M. Summers,et al.  A Bottom-Up Approach for Pancreas Segmentation Using Cascaded Superpixels and (Deep) Image Patch Labeling , 2015, IEEE Transactions on Image Processing.

[3]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[4]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[5]  Çağrı Kaymak,et al.  Semantic Image Segmentation for Autonomous Driving Using Fully Convolutional Networks , 2019, 2019 International Artificial Intelligence and Data Processing Symposium (IDAP).

[6]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[7]  Zhenyu Lin,et al.  An Improved Combination of Image Denoisers Using Spatial Local Fusion Strategy , 2020, IEEE Access.

[8]  Mario J. Pérez-Jiménez,et al.  An Overview of Hardware Implementation of Membrane Computing Models , 2020, ACM Comput. Surv..

[9]  Lin Lu,et al.  Pancreatic Segmentation via Ringed Residual U-Net , 2019, IEEE Access.

[10]  Yan Wang,et al.  A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans , 2016, MICCAI.

[11]  Gang Pan,et al.  Deep Attention Network for Egocentric Action Recognition , 2019, IEEE Transactions on Image Processing.

[12]  Le Lu,et al.  Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function , 2017, ArXiv.

[13]  Tuan D. Pham,et al.  DUNet: A deformable network for retinal vessel segmentation , 2018, Knowl. Based Syst..

[14]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[15]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[16]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  Alan L. Yuille,et al.  Recurrent Saliency Transformation Network: Incorporating Multi-stage Visual Cues for Small Organ Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Ajay Gulati,et al.  An adaptive hybrid technique for pancreas segmentation using CT image sequences , 2015, 2015 International Conference on Signal Processing, Computing and Control (ISPCC).

[20]  Danny Z. Chen,et al.  A Fully 3D Cascaded Framework for Pancreas Segmentation , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[21]  Shrish Verma,et al.  Deep Dilated V-Net for 3D Volume Segmentation of Pancreas in CT images , 2020, 2020 7th International Conference on Signal Processing and Integrated Networks (SPIN).

[22]  Yan Zhu,et al.  Densely connected deep U-Net for abdominal multi-organ segmentation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[23]  Hayaru Shouno,et al.  Analysis of function of rectified linear unit used in deep learning , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[24]  Yao Lu,et al.  RIC-Unet: An Improved Neural Network Based on Unet for Nuclei Segmentation in Histology Images , 2019, IEEE Access.

[25]  Alan L. Yuille,et al.  A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation , 2017, 2018 International Conference on 3D Vision (3DV).

[26]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[27]  Stephen Lin,et al.  Deformable ConvNets V2: More Deformable, Better Results , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Martin Styner,et al.  Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets , 2009, IEEE Transactions on Medical Imaging.

[29]  Guan Huang,et al.  Attention-Guided Unified Network for Panoptic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Ivan Tsmots,et al.  Hardware Implementation of Sigmoid Activation Functions using FPGA , 2019, 2019 IEEE 15th International Conference on the Experience of Designing and Application of CAD Systems (CADSM).

[31]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[32]  Shangqing Liu,et al.  Automatic Pancreas Segmentation via Coarse Location and Ensemble Learning , 2020, IEEE Access.

[33]  Ben Glocker,et al.  Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images , 2018, Medical Image Anal..

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  Fei Wu,et al.  Deep Q Learning Driven CT Pancreas Segmentation With Geometry-Aware U-Net , 2019, IEEE Transactions on Medical Imaging.

[36]  Yan Wang,et al.  Dual-Input Neural Network Integrating Feature Extraction and Deep Learning for Coronary Artery Disease Detection Using Electrocardiogram and Phonocardiogram , 2019, IEEE Access.

[37]  Lianfang Zheng,et al.  Mechanistic target of rapamycin in the tumor microenvironment and its potential as a therapeutic target for pancreatic cancer. , 2020, Cancer letters.

[38]  LinLin Shen,et al.  A Deep Residual Inception Network for HEp-2 Cell Classification , 2017, DLMIA/ML-CDS@MICCAI.

[39]  Răzvan Andonie,et al.  Hyperparameter optimization in learning systems , 2019, Journal of Membrane Computing.

[40]  Chengwen Chu,et al.  Multi‐atlas pancreas segmentation: Atlas selection based on vessel structure , 2017, Medical Image Anal..

[41]  K. Sheng,et al.  Feasibility of automated 3-dimensional magnetic resonance imaging pancreas segmentation , 2016, Advances in radiation oncology.

[42]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[44]  Goutam Sanyal,et al.  Guiding attention of faces through graph based visual saliency (GBVS) , 2019, Cognitive Neurodynamics.

[45]  Yike Guo,et al.  The assessment of small bowel motility with attentive deformable neural network , 2020, Inf. Sci..

[46]  Yanhong Zhou,et al.  Classification of ERP Signals from Mild Cognitive Impairment Patients with Diabetes using Dual Input Encoder Convolutional Neural Network , 2019, 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA).

[47]  Hexi Li,et al.  Recognition of Robot Based on Attention Mechanism and Convolutional Neural Network , 2019, 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[48]  Ying Wu,et al.  Automated segmentation of multiple sclerosis lesion subtypes with multichannel MRI , 2006, NeuroImage.

[49]  Krista A. Ehinger,et al.  A novel graph-based optimization framework for salient object detection , 2017, Pattern Recognit..

[51]  Yi Li,et al.  Deformable Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[52]  Stefan Wesarg,et al.  Automatic pancreas segmentation in contrast enhanced CT data using learned spatial anatomy and texture descriptors , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[53]  Syed Masroor Ali,et al.  Learning Optical Flow via Deformable Convolution and Feature Pyramid Networks , 2019, 2019 IEEE 7th International Conference on Computer Science and Network Technology (ICCSNT).

[54]  G. Hemalatha,et al.  Preprocessing techniques of facial image with Median and Gabor filters , 2016, 2016 International Conference on Information Communication and Embedded Systems (ICICES).

[55]  Asoke K. Nandi,et al.  Threshold algorithm for pancreas segmentation in Dixon water magnetic resonance images , 2017, 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[56]  A. Jemal,et al.  Cancer statistics, 2019 , 2019, CA: a cancer journal for clinicians.

[57]  Heasoo Hwang,et al.  A robust deep convolutional neural network with batch-weighted loss for heartbeat classification , 2019, Expert Syst. Appl..

[58]  Jong Chul Ye,et al.  Deep residual learning for compressed sensing MRI , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[59]  Hao Li,et al.  A Model-Driven Stack-Based Fully Convolutional Network for Pancreas Segmentation , 2019, 2020 5th International Conference on Communication, Image and Signal Processing (CCISP).

[60]  L. R. Dice Measures of the Amount of Ecologic Association Between Species , 1945 .