HRED-Net: High-Resolution Encoder-Decoder Network for Fine-Grained Image Segmentation

Accurate segmentation of fine-grained information is an important step in medical image analysis applications. With the development of the encoder-decoder-based networks, various network structures and algorithms have made significant progress in semantic segmentation tasks. This work aims to present a novel high-resolution encoder-decoder network (HRED-Net) for fine-grained image segmentation that is highly accurate for small-scale targets. We design a multiscale context connection module to extract feature information without reducing the resolution, and propose a multiresolution fusion model to fine-tune the final results. In addition, these modules are trained together with a detail-oriented loss function to enhance the model’s perception of fine-grained parts. Through experiments on the DRIVE dataset, we found a balance between these modules, and our comparison results show that in addition to the extraction multiscale features, the fusion of multiresolution prediction information is also beneficial for fine-grained segmentation. Our method yielded significant improvements in the accuracy and sensitivity in retinal vessel and lung segmentation tasks.

[1]  Song Guo,et al.  Deeply supervised neural network with short connections for retinal vessel segmentation , 2018, Int. J. Medical Informatics.

[2]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[3]  Matthew B. Blaschko,et al.  A Discriminatively Trained Fully Connected Conditional Random Field Model for Blood Vessel Segmentation in Fundus Images , 2017, IEEE Transactions on Biomedical Engineering.

[4]  Changxing Ding,et al.  Dual-force convolutional neural networks for accurate brain tumor segmentation , 2019, Pattern Recognit..

[5]  Philip H. S. Torr,et al.  Weakly- and Semi-Supervised Panoptic Segmentation , 2022 .

[6]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Tianfu Wang,et al.  A Cross-Modality Learning Approach for Vessel Segmentation in Retinal Images , 2016, IEEE Transactions on Medical Imaging.

[8]  Shuicheng Yan,et al.  A survey on deep learning-based fine-grained object classification and semantic segmentation , 2017, International Journal of Automation and Computing.

[9]  Hai Su,et al.  Fine-grained histopathological image analysis via robust segmentation and large-scale retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Hong Kang,et al.  BTS-DSN: Deeply supervised neural network with short connections for retinal vessel segmentation , 2019, Int. J. Medical Informatics.

[11]  Alejandro F. Frangi,et al.  Retinal Image Synthesis and Semi-Supervised Learning for Glaucoma Assessment , 2019, IEEE Transactions on Medical Imaging.

[12]  Frédo Durand,et al.  Data augmentation using learned transforms for one-shot medical image segmentation , 2019, ArXiv.

[13]  A.D. Hoover,et al.  Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response , 2000, IEEE Transactions on Medical Imaging.

[14]  Dinggang Shen,et al.  High-Resolution Encoder–Decoder Networks for Low-Contrast Medical Image Segmentation , 2020, IEEE Transactions on Image Processing.

[15]  Yujie Li,et al.  NAS-Unet: Neural Architecture Search for Medical Image Segmentation , 2019, IEEE Access.

[16]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Deepu Rajan,et al.  Backtracking ScSPM Image Classifier for Weakly Supervised Top-Down Saliency , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Wenqing Sun,et al.  Fine-grained lung nodule segmentation with pyramid deconvolutional neural network , 2019, Medical Imaging.

[22]  Adel Hafiane,et al.  Clustering initiated multiphase active contours and robust separation of nuclei groups for tissue segmentation , 2008, 2008 19th International Conference on Pattern Recognition.

[23]  Xiaoxiao Li,et al.  Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Christoph H. Lampert,et al.  Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation , 2016, ECCV.

[26]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[27]  Shenghua Gao,et al.  CE-Net: Context Encoder Network for 2D Medical Image Segmentation , 2019, IEEE Transactions on Medical Imaging.

[28]  Américo Oliveira,et al.  Retinal vessel segmentation based on Fully Convolutional Neural Networks , 2018, Expert Syst. Appl..

[29]  Tuan D. Pham,et al.  DUNet: A deformable network for retinal vessel segmentation , 2018, Knowl. Based Syst..

[30]  Krzysztof Krawiec,et al.  Segmenting Retinal Blood Vessels With Deep Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[31]  Erik J. Bekkers,et al.  Retinal vessel delineation using a brain-inspired wavelet transform and random forest , 2017, Pattern Recognit..

[32]  Christos D Giachritsis,et al.  Coarse-grained information dominates fine-grained information in judgments of time-to-contact from retinal flow , 2000, Vision Research.

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[35]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Abhishek Verma,et al.  New Deep Neural Nets for Fine-Grained Diabetic Retinopathy Recognition on Hybrid Color Space , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[37]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Juntang Zhuang,et al.  LadderNet: Multi-path networks based on U-Net for medical image segmentation , 2018, ArXiv.

[39]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Vijayan K. Asari,et al.  Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation , 2018, ArXiv.

[43]  René Vidal,et al.  End-to-End Fine-Grained Action Segmentation and Recognition Using Conditional Random Field Models and Discriminative Sparse Coding , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[44]  Xin Yang,et al.  Fine-Grained Recurrent Neural Networks for Automatic Prostate Segmentation in Ultrasound Images , 2016, AAAI.

[45]  P. Matthews,et al.  UK Biobank’s cardiovascular magnetic resonance protocol , 2015, Journal of Cardiovascular Magnetic Resonance.

[46]  Qi Tian,et al.  CcNet: A cross-connected convolutional network for segmenting retinal vessels using multi-scale features , 2020, Neurocomputing.

[47]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[48]  Nasir M. Rajpoot,et al.  Dual-Channel Active Contour Model for Megakaryocytic Cell Segmentation in Bone Marrow Trephine Histology Images , 2017, IEEE Transactions on Biomedical Engineering.

[49]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Daniel Racoceanu,et al.  Cell nuclei extraction from breast cancer histopathologyimages using colour, texture, scale and shape information , 2013, Diagnostic Pathology.

[51]  W. Marsden I and J , 2012 .

[52]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[53]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Shenghuo Zhu,et al.  Efficient Object Detection and Segmentation for Fine-Grained Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Dinggang Shen,et al.  Semantic-guided Encoder Feature Learning for Blurry Boundary Delineation , 2019, ArXiv.