Uncertainty-Aware Multi-resolution Whole-Body MR to CT Synthesis

Synthesising computed tomography (CT) images from magnetic resonance images (MRI) plays an important role in the field of medical image analysis, both for quantification and diagnostic purposes. Especially for brain applications, convolutional neural networks (CNNs) have proven to be a valuable tool in this image translation task, achieving state-of-the-art results. Full body image synthesis, however, remains largely uncharted territory, bearing many challenges including a limited field of view and large image size, complex spatial context and anatomical differences between time-elapsing image acquisitions. We propose a novel multi-resolution cascade 3D network for end-to-end full-body MR to CT synthesis. We show that our method outperforms popular CNNs like U-Net in 2D and 3D. We further propose to include uncertainty in our network as a measure of safety and to account for intrinsic noise and misalignment in the data.

[1]  Pawel Markiewicz,et al.  Deep Boosted Regression for MR to CT Synthesis , 2018, SASHIMI@MICCAI.

[2]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[3]  Ninon Burgos,et al.  Attenuation Correction Synthesis for Hybrid PET-MR Scanners: Application to Brain Studies , 2014, IEEE Transactions on Medical Imaging.

[4]  Sébastien Ourselin,et al.  Fast free-form deformation using graphics processing units , 2010, Comput. Methods Programs Biomed..

[5]  Shu Liao,et al.  Unpaired whole-body MR to CT synthesis with correlation coefficient constrained adversarial learning , 2019, Medical Imaging: Image Processing.

[6]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Ninon Burgos,et al.  A multi-centre evaluation of eleven clinically feasible brain PET/MRI attenuation correction techniques using a large cohort of patients , 2016, NeuroImage.

[8]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[9]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[10]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[11]  Ghassan Hamarneh,et al.  Multi-resolution-Tract CNN with Hybrid Pretrained and Skin-Lesion Trained Layers , 2016, MLMI@MICCAI.

[12]  Parashkev Nachev,et al.  Computer Methods and Programs in Biomedicine NiftyNet: a deep-learning platform for medical imaging , 2022 .

[13]  Aaron Carass,et al.  Validating Uncertainty in Medical Image Translation , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[14]  Zilei Wang,et al.  Weighted Channel Dropout for Regularization of Deep Convolutional Neural Network , 2019, AAAI.

[15]  Sébastien Ourselin,et al.  On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task , 2017, IPMI.

[16]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.