Learning to Predict Error for MRI Reconstruction

We present a novel perspective on uncertainty quantification in deep learning, based on the bias-variance decomposition of ensembles. We argue that the existing uncertainty estimation methods are suboptimal in estimating the prediction error of an ensemble, and propose a new two-stage procedure instead, where in the first stage we estimate the unknown function using an ensemble, and in the second stage we fit a separate neural net to the errors of the first model. We argue that this has several advantages, among which better control over regularization such as early stopping, and a more accurate approximation to the aleatoric uncertainty. We extensively test our method on both synthetic as well as real world MRI data, and find that our method significantly improves uncertainty estimates compared to various alternatives such as dropout and deep ensemble based methods.

[1]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[2]  Henkjan Huisman,et al.  Supervised Uncertainty Quantification for Segmentation with Multiple Annotations , 2019, MICCAI.

[3]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[4]  T. Davenport,et al.  The potential for artificial intelligence in healthcare , 2019, Future Healthcare Journal.

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  E. Iso,et al.  Measurement Uncertainty and Probability: Guide to the Expression of Uncertainty in Measurement , 1995 .

[7]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[8]  Kilian Q. Weinberger,et al.  On Calibration of Modern Neural Networks , 2017, ICML.

[9]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Finale Doshi-Velez,et al.  Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning , 2017, ICML.

[11]  Timo Kohlberger,et al.  Evaluating Segmentation Error without Ground Truth , 2012, MICCAI.

[12]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[13]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[14]  Antonio Criminisi,et al.  Bayesian Image Quality Transfer with CNNs: Exploring Uncertainty in dMRI Super-Resolution , 2017, MICCAI.

[15]  Myunghee Cho Paik,et al.  Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation , 2020, Comput. Stat. Data Anal..

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  A. Kiureghian,et al.  Aleatory or epistemic? Does it matter? , 2009 .

[18]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[19]  Konstantinos Kamnitsas,et al.  Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth , 2017, IEEE Transactions on Medical Imaging.

[20]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[21]  Stefano Ermon,et al.  Accurate Uncertainties for Deep Learning Using Calibrated Regression , 2018, ICML.

[22]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[23]  A. Weigend,et al.  Estimating the mean and variance of the target probability distribution , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[24]  Pascal Vincent,et al.  fastMRI: An Open Dataset and Benchmarks for Accelerated MRI , 2018, ArXiv.

[25]  Jonas Adler,et al.  Deep Bayesian Inversion , 2018, ArXiv.

[26]  Max Welling,et al.  i-RIM applied to the fastMRI challenge , 2019, ArXiv.

[27]  Sebastian Nowozin,et al.  Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift , 2019, NeurIPS.

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..