Deep Semantic Segmentation of Kidney and Space-Occupying Lesion Area Based on SCNN and ResNet Models Combined with SIFT-Flow Algorithm

Renal segmentation is one of the most fundamental and challenging task in computer aided diagnosis systems. In order to overcome the shortcomings of automatic kidney segmentation based on deep network for abdominal CT images, a two-stage semantic segmentation of kidney and space-occupying lesion area based on SCNN and ResNet models combined with SIFT-flow transformation is proposed in paper, which is divided into two stages: image retrieval and semantic segmentation. To facilitate the image retrieval, a metric learning-based approach is firstly adopted to construct a deep convolutional neural network structure using SCNN and ResNet network to extract image features and minimize the impact of interference factors on features, so as to obtain the ability to represent the abdominal CT scan image with the same angle under different imaging conditions. And then, SIFT Flow transformation is introduced, which adopts MRF to fuse label information, priori spatial information and smoothing information to establish the dense matching relationship of pixels so that the semantics can be transferred from the known image to the target image so as to obtain the semantic segmentation result of kidney and space-occupying lesion area. In order to validate effectiveness and efficiency of our proposed method, we conduct experiments on self-establish CT dataset, focus on kidney organ and most of which have tumors inside of the kidney, and abnormal deformed shape of kidney. The experimental results qualitatively and quantitatively show that the accuracy of kidney segmentation is greatly improved, and the key information of the proportioned tumor occupying a small area of the image are exhibited a good segmentation results. In addition, our algorithm has also achieved ideal results in the clinical verification, which is suitable for intelligent medicine equipment applications.

[1]  Lubomir M. Hadjiiski,et al.  Bladder Cancer Segmentation in CT for Treatment Response Assessment: Application of Deep-Learning Convolution Neural Network—A Pilot Study , 2016, Tomography.

[2]  Seyed-Ahmad Ahmadi,et al.  Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields , 2016, MICCAI.

[3]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[4]  Yaozong Gao,et al.  Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching , 2017, Deep Learning for Medical Image Analysis.

[5]  Jaime S. Cardoso,et al.  Deep Learning and Data Labeling for Medical Applications , 2016, Lecture Notes in Computer Science.

[6]  Nima Tajbakhsh,et al.  A Comprehensive Computer-Aided Polyp Detection System for Colonoscopy Videos , 2015, IPMI.

[7]  Daguang Xu,et al.  Robust 3D Organ Localization with Dual Learning Architectures and Fusion , 2016, LABELS/DLMIA@MICCAI.

[8]  Hiroyuki Yoshida,et al.  Deep transfer learning of virtual endoluminal views for the detection of polyps in CT colonography , 2016, SPIE Medical Imaging.

[9]  Hao Chen,et al.  Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection , 2017, IEEE Transactions on Biomedical Engineering.

[10]  Dinggang Shen,et al.  MRI‐based prostate cancer detection with high‐level representation and hierarchical classification , 2017, Medical physics.

[11]  Carmen C. Y. Poon,et al.  Automatic Detection and Classification of Colorectal Polyps by Transferring Low-Level CNN Features From Nonmedical Domain , 2017, IEEE Journal of Biomedical and Health Informatics.

[12]  Hao Chen,et al.  Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images , 2017, AAAI.

[13]  L. Joskowicz,et al.  Automatic liver tumor segmentation in follow-up CT studies using Convolutional Neural Networks , 2015 .

[14]  Shu Liao,et al.  Representation Learning: A Unified Deep Learning Framework for Automatic Prostate MR Segmentation , 2013, MICCAI.

[15]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[16]  Hayit Greenspan,et al.  Fully Convolutional Network for Liver Segmentation and Lesions Detection , 2016, LABELS/DLMIA@MICCAI.

[17]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[18]  Leo Joskowicz,et al.  Automatic Liver Tumor Segmentation in Follow-Up CT Scans: Preliminary Method and Results , 2015, Patch-MI@MICCAI.

[19]  Hiroyuki Yoshida,et al.  Deep learning for electronic cleansing in dual-energy CT colonography , 2016, SPIE Medical Imaging.

[20]  Ronald M. Summers,et al.  A Bottom-Up Approach for Pancreas Segmentation Using Cascaded Superpixels and (Deep) Image Patch Labeling , 2015, IEEE Transactions on Image Processing.

[21]  Purang Abolmaesumi,et al.  Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study , 2016, International Journal of Computer Assisted Radiology and Surgery.

[22]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[23]  Fang Lu,et al.  Automatic 3D liver location and segmentation via convolutional neural network and graph cut , 2016, International Journal of Computer Assisted Radiology and Surgery.

[24]  Feng Chen,et al.  Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets , 2016, International Journal of Computer Assisted Radiology and Surgery.

[25]  Hariharan Ravishankar,et al.  Understanding the Mechanisms of Deep Transfer Learning for Medical Images , 2016, LABELS/DLMIA@MICCAI.

[26]  Jialin Peng,et al.  Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution , 2016, Physics in medicine and biology.

[27]  Ronald M. Summers,et al.  Colitis detection on computed tomography using regional convolutional neural networks , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[28]  Nassir Navab,et al.  Deeply learnt hashing forests for content based image retrieval in prostate MR images , 2016, SPIE Medical Imaging.

[29]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[30]  Fucang Jia,et al.  Automatic Segmentation of Liver Tumor in CT Images with Deep Convolutional Neural Networks , 2015 .

[31]  Christopher Joseph Pal,et al.  Convolutional networks for kidney segmentation in contrast-enhanced CT scans , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[32]  Le Lu,et al.  Pancreas Segmentation in MRI Using Graph-Based Decision Fusion on Convolutional Neural Networks , 2016, MICCAI.

[33]  Ronald M. Summers,et al.  Spatial Aggregation of Holistically-Nested Networks for Automated Pancreas Segmentation , 2016, MICCAI.