Collaborative Boundary-aware Context Encoding Networks for Error Map Prediction

Medical image segmentation is usually regarded as one of the most important intermediate steps in clinical situations and medical imaging research. Thus, accurately assessing the segmentation quality of the automatically generated predictions is essential for guaranteeing the reliability of the results of the computer-assisted diagnosis (CAD). Many researchers apply neural networks to train segmentation quality regression models to estimate the segmentation quality of a new data cohort without labeled ground truth. Recently, a novel idea is proposed that transforming the segmentation quality assessment (SQA) problem intothe pixel-wise error map prediction task in the form of segmentation. However, the simple application of vanilla segmentation structures in medical image fails to detect some small and thin error regions of the auto-generated masks with complex anatomical structures. In this paper, we propose collaborative boundaryaware context encoding networks called AEP-Net for error prediction task. Specifically, we propose a collaborative feature transformation branch for better feature fusion between images and masks, and precise localization of error regions. Further, we propose a context encoding module to utilize the global predictor from the error map to enhance the feature representation and regularize the networks. We perform experiments on IBSR v2.0 dataset and ACDC dataset. The AEP-Net achieves an average DSC of 0.8358, 0.8164 for error prediction task,and shows a high Pearson correlation coefficient of 0.9873 between the actual segmentation accuracy and the predicted accuracy inferred from the predicted error map on IBSR v2.0 dataset, which verifies the efficacy of our AEP-Net.

[1]  Dinggang Shen,et al.  Contour Knowledge Transfer for Salient Object Detection , 2018, ECCV.

[2]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[3]  Konstantinos Kamnitsas,et al.  Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth , 2017, IEEE Transactions on Medical Imaging.

[4]  Daniel Rueckert,et al.  Deep Generative Model-based Quality Control for Cardiac MRI Segmentation , 2020, MICCAI.

[5]  Wenxiang Deng,et al.  Robust Image Segmentation Quality Assessment , 2019 .

[6]  Khan Muhammad,et al.  Multi-Class Skin Lesion Detection and Classification via Teledermatology , 2021, IEEE Journal of Biomedical and Health Informatics.

[7]  M. Sharif,et al.  Skin Lesion Segmentation and Multiclass Classification Using Deep Learning Features and Improved Moth Flame Optimization , 2021, Diagnostics.

[8]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[9]  Penglang Shui,et al.  Color edge detection by learning classification network with anisotropic directional derivative matrices , 2021, Pattern Recognit..

[10]  Jian Sun,et al.  ExFuse: Enhancing Feature Fusion for Semantic Segmentation , 2018, ECCV.

[11]  Hanqing Lu,et al.  Contextual deconvolution network for semantic segmentation , 2020, Pattern Recognit..

[12]  Muhammad Attique Khan,et al.  Pixels to Classes: Intelligent Learning Framework for Multiclass Skin Lesion Localization and Classification , 2021, Comput. Electr. Eng..

[13]  M. Sharif,et al.  A Hierarchical Three-Step Superpixels and Deep Learning Framework for Skin Lesion Classification. , 2021, Methods.

[14]  Muhammad Attique Khan,et al.  Computer Decision Support System for Skin Cancer Localization and Classification , 2021 .

[15]  Rongzhao Zhang,et al.  A Fine-Grain Error Map Prediction and Segmentation Quality Assessment Framework for Whole-Heart Segmentation , 2019, MICCAI.

[16]  Xiaogang Wang,et al.  Context Encoding for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Jonathan T. Barron,et al.  Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[19]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[20]  Matthieu Cord,et al.  Addressing Failure Prediction by Learning Model Confidence , 2019, NeurIPS.

[21]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Ben Glocker,et al.  Real-time Prediction of Segmentation Quality , 2018, MICCAI.

[23]  Pedro Costa,et al.  A No-Reference Quality Metric for Retinal Vessel Tree Segmentation , 2018, MICCAI.

[24]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[25]  Jianbo Shi,et al.  Semantic Segmentation with Boundary Neural Fields , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[27]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[28]  Lingqiao Liu,et al.  Boundarymix: Generating pseudo-training images for improving segmentation with scribble annotations , 2021, Pattern Recognit..

[29]  Syed Rameez Naqvi,et al.  Human gait analysis for osteoarthritis prediction: a framework of deep learning and kernel extreme learning machine , 2021, Complex & Intelligent Systems.

[30]  Jie Li,et al.  A Multi-resolution Coarse-to-Fine Segmentation Framework with Active Learning in 3D Brain MRI , 2019, IScIDE.

[31]  Fei Xu,et al.  Automatic Breast Ultrasound Image Segmentation: A Survey , 2017, Pattern Recognit..

[32]  Yukun Cao,et al.  Learning Directional Feature Maps for Cardiac MRI Segmentation , 2020, MICCAI.

[33]  Myunghee Cho Paik,et al.  Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation , 2020, Comput. Stat. Data Anal..

[34]  Dimitris N. Metaxas,et al.  Few-Shot Learning by a Cascaded Framework With Shape-Constrained Pseudo Label Assessment for Whole Heart Segmentation , 2021, IEEE Transactions on Medical Imaging.

[35]  Nassir Navab,et al.  Inherent Brain Segmentation Quality Control from Fully ConvNet Monte Carlo Sampling , 2018, MICCAI.

[36]  Dong Yang,et al.  An Alarm System for Segmentation Algorithm Based on Shape Model , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Sanja Fidler,et al.  Gated-SCNN: Gated Shape CNNs for Semantic Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[38]  Sergio Escalera,et al.  Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge , 2021, IEEE Transactions on Medical Imaging.

[39]  Hao Chen,et al.  VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images , 2017, NeuroImage.

[40]  Timo Kohlberger,et al.  Evaluating Segmentation Error without Ground Truth , 2012, MICCAI.

[41]  Chunna Tian,et al.  Quality-driven deep active learning method for 3D brain MRI segmentation , 2021, Neurocomputing.

[42]  Wenyu Liu,et al.  Boundary-preserving Mask R-CNN , 2020, ECCV.

[43]  Ming-Yu Liu,et al.  CASENet: Deep Category-Aware Semantic Edge Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).