Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation

Fine renal artery segmentation on abdominal CT angiography (CTA) image is one of the most important tasks for kidney disease diagnosis and pre-operative planning. It will help clinicians locate each interlobar artery's blood-feeding region via providing the complete 3D renal artery tree masks. However, it is still a task of great challenges due to the large intra-scale changes, large inter-anatomy variation, thin structures, small volume ratio and small labeled dataset of the fine renal artery. In this paper, we propose the first semi-supervised 3D fine renal artery segmentation framework, DPA-DenseBiasNet, which combines deep prior anatomy (DPA), dense biased network (DenseBiasNet) and hard region adaptation loss (HRA): 1) Based on our proposed dense biased connection, the DenseBiasNet fuses multi-receptive field and multi-resolution feature maps for large intra-scale changes. This dense biased connection also obtains a dense information flow and dense gradient flow so that the training is accelerated and the accuracy is enhanced. 2) DPA features extracted from an autoencoder (AE) are embedded in DenseBiasNet to cope with the challenge of large inter-anatomy variation and thin structures. The AE is pre-trained (unsupervised) by numerous unlabeled data to achieve the representation ability of anatomy features and these features are embedded in DenseBiasNet. This process will not introduce incorrect labels as optimization targets and thus contributes to a stable semi-supervised training strategy that is suitable for sensitive thin structures. 3) The HRA selects the loss value calculation region dynamically according to the segmentation quality so the network will pay attention to the hard regions in the training process and keep the class balanced. Experiments demonstrated that DPA-DenseBiasNet had high predictive accuracy and generalization with the Dice coefficient of 0.884 which increased by 0.083 compared with 3D U-Net (Çiçek et al., 2016). This revealed our framework with great potential for the 3D fine renal artery segmentation in clinical practice.

[1]  Lorenzo Marconi,et al.  EAU guidelines on renal cell carcinoma: 2014 update. , 2010, European urology.

[2]  Tapani Raiko,et al.  Lateral Connections in Denoising Autoencoders Support Supervised Learning , 2015, ArXiv.

[3]  A. Jemal,et al.  Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries , 2018, CA: a cancer journal for clinicians.

[4]  Jing Liu,et al.  Discrimination-aware Channel Pruning for Deep Neural Networks , 2018, NeurIPS.

[5]  Tao Zhao,et al.  Segmentation of Renal Structures for Image-Guided Surgery , 2018, MICCAI.

[6]  Kun Yu,et al.  DenseASPP for Semantic Segmentation in Street Scenes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Yang Chen,et al.  Stereo-Correlation and Noise-Distribution Aware ResVoxGAN for Dense Slices Reconstruction and Noise Reduction in Thick Low-Dose CT , 2019, MICCAI.

[8]  Xiantong Zhen,et al.  Supervised descriptor learning for multi-output regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[10]  Tapani Raiko,et al.  Semi-supervised Learning with Ladder Networks , 2015, NIPS.

[11]  Hossein Mobahi,et al.  Deep Learning via Semi-supervised Embedding , 2012, Neural Networks: Tricks of the Trade.

[12]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[13]  Isabelle Bloch,et al.  A review of 3D vessel lumen segmentation techniques: Models, features and extraction schemes , 2009, Medical Image Anal..

[14]  Jeroen J. Bax,et al.  Automatic centerline extraction of coronary arteries in coronary computed tomographic angiography , 2011, The International Journal of Cardiovascular Imaging.

[15]  Dwarikanath Mahapatra,et al.  Semi-supervised Segmentation of Optic Cup in Retinal Fundus Images Using Variational Autoencoder , 2017, MICCAI.

[16]  Milan Sonka,et al.  Intrathoracic airway trees: segmentation and airway morphology analysis from low-dose CT scans , 2005, IEEE Transactions on Medical Imaging.

[17]  Jordi Vitrià,et al.  Tracking elongated structures using statistical snakes , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Bo Wang,et al.  Deep Co-Training for Semi-Supervised Image Recognition , 2018, ECCV.

[19]  Francesco Porpiglia,et al.  Hyperaccuracy Three-dimensional Reconstruction Is Able to Maximize the Efficacy of Selective Clamping During Robot-assisted Partial Nephrectomy for Complex Renal Masses. , 2018, European urology.

[20]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[21]  Daniël M Pelt,et al.  A mixed-scale dense convolutional neural network for image analysis , 2017, Proceedings of the National Academy of Sciences.

[22]  Chao Qin,et al.  Laparoscopic partial nephrectomy with segmental renal artery clamping: technique and clinical outcomes. , 2011, European urology.

[23]  Yaozong Gao,et al.  ASDNet: Attention Based Semi-supervised Deep Networks for Medical Image Segmentation , 2018, MICCAI.

[24]  Jiasong Wu,et al.  DPA-DenseBiasNet: Semi-supervised 3D Fine Renal Artery Segmentation with Dense Biased Network and Deep Priori Anatomy , 2019, MICCAI.

[25]  Nassir Navab,et al.  Semi-supervised Deep Learning for Fully Convolutional Networks , 2017, MICCAI.

[26]  Huazhong Shu,et al.  Automatic kidney segmentation in CT images based on multi-atlas image registration , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[27]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Bordei Petru,et al.  Morphological assessments on the arteries of the superior renal segment , 2012, Surgical and Radiologic Anatomy.

[29]  Theo van Walsum,et al.  SEMI-AUTOMATIC CORONARY ARTERY CENTERLINE EXTRACTION IN COMPUTED TOMOGRAPHY ANGIOGRAPHY DATA , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[30]  Tao Zhao,et al.  Kid-Net: Convolution Networks for Kidney Vessels Segmentation from CT-Volumes , 2018, MICCAI.

[31]  Zhiqiang Shen,et al.  Learning Efficient Convolutional Networks through Network Slimming , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[33]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[34]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[36]  Badrinath Roysam,et al.  Robust 3-D Modeling of Vasculature Imagery Using Superellipsoids , 2007, IEEE Transactions on Medical Imaging.

[37]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[38]  Xiangyu Zhang,et al.  ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design , 2018, ECCV.

[39]  Chuhan Wu,et al.  Semi-supervised dimensional sentiment analysis with variational autoencoder , 2019, Knowl. Based Syst..

[40]  Laurent D. Cohen,et al.  Deformable tree models for 2D and 3D branching structures extraction , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[41]  Max A. Viergever,et al.  Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks , 2016, Medical Image Anal..

[42]  Ming-Hsuan Yang,et al.  Adversarial Learning for Semi-supervised Semantic Segmentation , 2018, BMVC.

[43]  Jun Wu,et al.  A semi-supervised deep learning method based on stacked sparse auto-encoder for cancer prediction using RNA-seq data , 2018, Comput. Methods Programs Biomed..

[44]  Terry M. Peters,et al.  Global Assessment of Cardiac Function Using Image Statistics in MRI , 2012, MICCAI.

[45]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[46]  Daniel Rueckert,et al.  Recurrent neural networks for aortic image sequence segmentation with sparse annotations , 2018, MICCAI.

[47]  Tian-Yu Liu,et al.  EasyEnsemble and Feature Selection for Imbalance Data Sets , 2009, 2009 International Joint Conference on Bioinformatics, Systems Biology and Intelligent Computing.

[48]  Plamen P. Angelov,et al.  Semi-supervised deep rule-based approach for image classification , 2018, Appl. Soft Comput..

[49]  Guanyu Yang,et al.  Application of a Functional3-dimensional Perfusion Model in Laparoscopic Partial Nephrectomy With Precise Segmental Renal Artery Clamping. , 2019, Urology.

[50]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[51]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[52]  Ben Glocker,et al.  Semi-supervised Learning for Network-Based Cardiac MR Image Segmentation , 2017, MICCAI.

[53]  Lizhe Wang,et al.  A semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[54]  Bernhard Preim,et al.  Geometrical and Structural Analysis of Vessel Systems in 3D Medical Image Datasets , 2004 .

[55]  Yuxing Tang,et al.  Visual and Semantic Knowledge Transfer for Large Scale Semi-Supervised Object Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Wufeng Xue,et al.  Direct Estimation of Regional Wall Thicknesses via Residual Recurrent Neural Network , 2017, IPMI.

[57]  Jelmer M. Wolterink,et al.  Automatic segmentation of thoracic aorta segments in low-dose chest CT , 2018, Medical Imaging.

[58]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[59]  Max A. Viergever,et al.  Coronary artery centerline extraction in cardiac CT angiography using a CNN‐based orientation classifier , 2018, Medical Image Anal..

[60]  Heye Zhang,et al.  PV-LVNet: Direct left ventricle multitype indices estimation from 2D echocardiograms of paired apical views with deep neural networks , 2019, Medical Image Anal..

[61]  Friedhelm Schwenker,et al.  Semi-supervised Learning , 2013, Handbook on Neural Information Processing.

[62]  Limin Luo,et al.  K-Net: Integrate Left Ventricle Segmentation and Direct Quantification of Paired Echo Sequence , 2019, IEEE Transactions on Medical Imaging.

[63]  P. Shao,et al.  Precise segmental renal artery clamping under the guidance of dual-source computed tomography angiography during laparoscopic partial nephrectomy. , 2012, European urology.

[64]  Elena De Momi,et al.  Blood vessel segmentation algorithms - Review of methods, datasets and evaluation metrics , 2018, Comput. Methods Programs Biomed..