Inconsistency-Aware Uncertainty Estimation for Semi-Supervised Medical Image Segmentation

In semi-supervised medical image segmentation, most previous works draw on the common assumption that higher entropy means higher uncertainty. In this paper, we investigate a novel method of estimating uncertainty. We observe that, when assigned different misclassification costs in a certain degree, if the segmentation result of a pixel becomes inconsistent, this pixel shows a relative uncertainty in its segmentation. Therefore, we present a new semi-supervised segmentation model, namely, conservative-radical network (CoraNet in short) based on our uncertainty estimation and separate self-training strategy. In particular, our CoraNet model consists of three major components: a conservative-radical module (CRM), a certain region segmentation network (C-SN), and an uncertain region segmentation network (UC-SN) that could be alternatively trained in an end-to-end manner. We have extensively evaluated our method on various segmentation tasks with publicly available benchmark datasets, including CT pancreas, MR endocardium, and MR multi-structures segmentation on the ACDC dataset. Compared with the current state of the art, our CoraNet has demonstrated superior performance. In addition, we have also analyzed its connection with and difference from conventional methods of uncertainty estimation in semi-supervised medical image segmentation.

[1]  Patrick Pérez,et al.  ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Christian Desrosiers,et al.  Boosting Semi-supervised Image Segmentation with Global and Local Mutual Information Regularization , 2021, ArXiv.

[3]  Shuai Chen,et al.  Multi-Task Attention-Based Semi-Supervised Learning for Medical Image Segmentation , 2019, MICCAI.

[4]  Yinghuan Shi,et al.  Does Manual Delineation only Provide the Side Information in CT Prostate Segmentation? , 2017, MICCAI.

[5]  Hao Chen,et al.  Transformation-Consistent Self-Ensembling Model for Semisupervised Medical Image Segmentation , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[6]  M. Jorge Cardoso,et al.  Towards safe deep learning: accurately quantifying biomarker uncertainty in neural network predictions , 2018, MICCAI.

[7]  Nima Tajbakhsh,et al.  UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[8]  Dong-Hyun Lee,et al.  Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks , 2013 .

[9]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[10]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[11]  Sebastian J. Schlecht,et al.  Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks , 2017, ArXiv.

[12]  Yinghuan Shi,et al.  Crossbar-Net: A Novel Convolutional Neural Network for Kidney Tumor Segmentation in CT Images , 2018, IEEE Transactions on Image Processing.

[13]  Mauricio Reyes,et al.  Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation , 2019, MICCAI.

[14]  Sébastien Ourselin,et al.  Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks , 2018, Neurocomputing.

[15]  Bryan M. Williams,et al.  Learning Active Contour Models for Medical Image Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Septimiu E. Salcudean,et al.  Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[17]  James S. Duncan,et al.  A Semi-supervised Joint Network for Simultaneous Left Ventricular Motion Tracking and Segmentation in 4D Echocardiography , 2020, MICCAI.

[18]  Yinghuan Shi,et al.  Crossover-Net: Leveraging vertical-horizontal crossover relation for robust medical image segmentation , 2020, Pattern Recognit..

[19]  Julien Cornebise,et al.  Weight Uncertainty in Neural Network , 2015, ICML.

[20]  Tanveer F. Syeda-Mahmood,et al.  3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes , 2018, MICCAI.

[21]  Wu-Jun Li,et al.  DMNet: Difference Minimization Network for Semi-supervised Segmentation in Medical Images , 2020, MICCAI.

[22]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[23]  Yinan Chen,et al.  Semi-supervised Medical Image Segmentation through Dual-task Consistency , 2020, ArXiv.

[24]  John K. Tsotsos,et al.  Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI , 2008, Medical Image Anal..

[25]  Marc Combalia,et al.  Uncertainty Estimation in Deep Neural Networks for Dermoscopic Image Classification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Lin Yang,et al.  Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images , 2017, MICCAI.

[27]  Quoc V. Le,et al.  Randaugment: Practical automated data augmentation with a reduced search space , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[28]  C. V. Jawahar,et al.  Universal Semi-Supervised Semantic Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[29]  Zhongchao Shi,et al.  Double-Uncertainty Weighted Method for Semi-supervised Learning , 2020, MICCAI.

[30]  Hao Chen,et al.  VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images , 2017, NeuroImage.

[31]  Xuming He,et al.  Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images , 2020, MICCAI.

[32]  Zhenyu Liu,et al.  Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation , 2017, Medical Image Anal..

[33]  Jizong Peng,et al.  Deep Co-Training for Semi-Supervised Image Segmentation , 2019, Pattern Recognit..

[34]  Fei Wu,et al.  Deep Q Learning Driven CT Pancreas Segmentation With Geometry-Aware U-Net , 2019, IEEE Transactions on Medical Imaging.

[35]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[36]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[37]  David Berthelot,et al.  MixMatch: A Holistic Approach to Semi-Supervised Learning , 2019, NeurIPS.

[38]  Jing Qin,et al.  OFF-eNET: An Optimally Fused Fully End-to-End Network for Automatic Dense Volumetric 3D Intracranial Blood Vessels Segmentation , 2020, IEEE Transactions on Image Processing.

[39]  Shenghua Gao,et al.  CE-Net: Context Encoder Network for 2D Medical Image Segmentation , 2019, IEEE Transactions on Medical Imaging.

[40]  C. Hudelot,et al.  Semi-Supervised Semantic Segmentation With Cross-Consistency Training , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Ryan P. Adams,et al.  Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks , 2015, ICML.

[42]  Daniel Rueckert,et al.  Recurrent neural networks for aortic image sequence segmentation with sparse annotations , 2018, MICCAI.

[43]  Chi-Wing Fu,et al.  Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation , 2019, MICCAI.

[44]  Shin Ishii,et al.  Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Philipp Berens,et al.  Test-time Data Augmentation for Estimation of Heteroscedastic Aleatoric Uncertainty in Deep Neural Networks , 2018 .

[46]  Max Welling,et al.  Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors , 2016, ICML.

[47]  Dinggang Shen,et al.  Global Deep Learning Methods for Multimodality Isointense Infant Brain Image Segmentation , 2018, ArXiv.

[48]  Pingkun Yan,et al.  Feature Fusion Encoder Decoder Network for Automatic Liver Lesion Segmentation , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[49]  Jun Li,et al.  Probability Map Guided Bi-directional Recurrent UNet for Pancreas Segmentation , 2019, ArXiv.

[50]  Dong Yang,et al.  Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation , 2020, Medical Image Anal..

[51]  David Berthelot,et al.  FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence , 2020, NeurIPS.

[52]  Timo Aila,et al.  Temporal Ensembling for Semi-Supervised Learning , 2016, ICLR.

[53]  Hao Chen,et al.  Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation , 2020, MICCAI.

[54]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[55]  Liang Chen,et al.  DRINet for Medical Image Segmentation , 2018, IEEE Transactions on Medical Imaging.

[56]  Jizong Peng,et al.  Mutual information deep regularization for semi-supervised segmentation , 2020, MIDL.

[57]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[58]  Harri Valpola,et al.  Weight-averaged consistency targets improve semi-supervised deep learning results , 2017, ArXiv.

[59]  Quoc V. Le,et al.  AutoAugment: Learning Augmentation Strategies From Data , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Guisheng Wang,et al.  A Teacher-Student Framework for Semi-supervised Medical Image Segmentation From Mixed Supervision , 2020, ArXiv.

[61]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Wengang Zhou,et al.  ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Image Segmentation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).