Task-based evaluation of deep image super-resolution in medical imaging

In medical imaging, it is sometimes desirable to acquire high resolution images that reveal anatomical and physiological information to support clinical practice such as diagnosis and image-guided therapies. However, for certain imaging modalities (i.e., magnetic resonance imaging (MRI)), acquiring high resolution images can be a very time-consuming and resource-intensive process. One popular solution recently developed is to create a high resolution version of the acquired low-resolution image by use of deep image super-resolution (DL-SR) methods. It has been demonstrated in literature that deep super-resolution networks can improve the image quality measured by traditional physical metrics such as mean square error (MSE), structural similarity index metric (SSIM) and peak signal-to-noise ratio (PSNR). However, it is not clear how well these metrics quantify the diagnostic value of the generated SR images. Here, a task-based super-resolution (SR) image quality assessment is conducted to quantitatively evaluate the efficiency and performance of DL-SR methods. A Rayleigh task is designed to investigate the impact of signal length and super-resolution network complexity on s binary detection performance. Numerical observers (NOs) including the regularized Hotelling Observer (RHO), the anthropomorphic Gabor channelized observers (Gabor CHO) and the ResNet-approximated ideal observer (ResNet-IO) are implemented to assess the Rayleigh task performance. For the datasets considered in this study, little to no improvement in task performance of the considered NOs due to the considered DL-SR SR networks, despite substantial improvement in traditional IQ metrics.

[1]  Guang Li,et al.  CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE) , 2018, IEEE Transactions on Medical Imaging.

[2]  Hayit Greenspan,et al.  Super-Resolution in Medical Imaging , 2009, Comput. J..

[3]  Eric Van Reeth,et al.  Super-resolution in magnetic resonance imaging: A review , 2012 .

[4]  Jeff Wood,et al.  Super‐resolution musculoskeletal MRI using deep learning , 2018, Magnetic resonance in medicine.

[5]  Yan Zhao,et al.  MRI image synthesis with dual discriminator adversarial learning and difficulty-aware attention mechanism for hippocampal subfields segmentation , 2020, Comput. Medical Imaging Graph..

[6]  Jie Yao,et al.  Predicting human performance by a channelized Hotelling observer model , 1992, Optics & Photonics.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Gözde B. Ünal,et al.  Semi-automated detection of anterior cruciate ligament injury from MRI , 2017, Comput. Methods Programs Biomed..

[10]  Kyle J. Myers,et al.  Rayleigh Task Performance as a Method to Evaluate Image Reconstruction Algorithms , 1991 .

[11]  Lee W Goldman,et al.  Principles of CT: Radiation Dose and Image Quality* , 2007, Journal of Nuclear Medicine Technology.

[12]  Melanie Grunwald,et al.  Foundations Of Image Science , 2016 .

[13]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Mark A. Anastasio,et al.  On Hallucinations in Tomographic Image Reconstruction , 2020, IEEE Transactions on Medical Imaging.

[15]  Xiaochuan Pan,et al.  Task-based optimization of dedicated breast CT via Hotelling observer metrics. , 2014, Medical physics.

[16]  Hua Li,et al.  Approximating the Ideal Observer and Hotelling Observer for Binary Signal Detection Tasks by Use of Supervised Learning Methods , 2019, IEEE Transactions on Medical Imaging.