SMORE: A Self-Supervised Anti-Aliasing and Super-Resolution Algorithm for MRI Using Deep Learning

High resolution magnetic resonance (MR) images are desired in many clinical and research applications. Acquiring such images with high signal-to-noise (SNR), however, can require a long scan duration, which is difficult for patient comfort, is more costly, and makes the images susceptible to motion artifacts. A very common practical compromise for both 2D and 3D MR imaging protocols is to acquire volumetric MR images with high in-plane resolution, but lower through-plane resolution. In addition to having poor resolution in one orientation, 2D MRI acquisitions will also have aliasing artifacts, which further degrade the appearance of these images. This paper presents an approach SMORE Code is available upon request. based on convolutional neural networks (CNNs) that restores image quality by improving resolution and reducing aliasing in MR imagesThe aliasing effect we study in this paper arises from undersampling in image space (via thick slice separation), not from undersampling in k-space.. This approach is self-supervised, which requires no external training data because the high-resolution and low-resolution data that are present in the image itself are used for training. For 3D MRI, the method consists of only one self-supervised super-resolution (SSR) deep CNN that is trained from the volumetric image data. For 2D MRI, there is a self-supervised anti-aliasing (SAA) deep CNN that precedes the SSR CNN, also trained from the volumetric image data. Both methods were evaluated on a broad collection of MR data, including filtered and downsampled images so that quantitative metrics could be computed and compared, and actual acquired low resolution images for which visual and sharpness measures could be computed and compared. The super-resolution method is shown to be visually and quantitatively superior to previously reported methods.

[1]  John F. Roddick,et al.  Sparse representation-based MRI super-resolution reconstruction , 2014 .

[2]  Xavier Bresson,et al.  An efficient total variation algorithm for super-resolution in fetal brain MRI with adaptive regularization , 2015, NeuroImage.

[3]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Feng Shi,et al.  Brain MRI super resolution using 3D deep densely connected neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[5]  Maxime Descoteaux,et al.  Collaborative patch-based super-resolution for diffusion-weighted images , 2013, NeuroImage.

[6]  Mohsen Guizani,et al.  Super-Resolution of Brain MRI Images Using Overcomplete Dictionaries and Nonlocal Similarity , 2019, IEEE Access.

[7]  Eugene W. Myers,et al.  Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks , 2017, MICCAI.

[8]  Amod Jog,et al.  Pulse Sequence Resilient Fast Brain Segmentation , 2018, MICCAI.

[9]  Aaron Carass,et al.  Self super-resolution for magnetic resonance images using deep networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[10]  Aaron Carass,et al.  Applications of a deep learning method for anti-aliasing and super-resolution in MRI. , 2019, Magnetic resonance imaging.

[11]  Peter A. Calabresi,et al.  A Deep Learning Based Anti-aliasing Self Super-Resolution Algorithm for MRI , 2018, MICCAI.

[12]  Antonio Criminisi,et al.  Image Quality Transfer via Random Forest Regression: Applications in Diffusion MRI , 2014, MICCAI.

[13]  Dwarikanath Mahapatra,et al.  Image Super Resolution Using Generative Adversarial Networks and Local Saliency Maps for Retinal Image Analysis , 2017, MICCAI.

[14]  Guillermo Sapiro,et al.  Hand-Held Video Deblurring Via Efficient Fourier Aggregation , 2015, IEEE Transactions on Computational Imaging.

[15]  M. Bernstein,et al.  Effect of windowing and zero‐filled reconstruction of MRI data on spatial resolution and acquisition strategy , 2001, Journal of magnetic resonance imaging : JMRI.

[16]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[17]  Zhongshi He,et al.  Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network , 2020, Neurocomputing.

[18]  Wiro J Niessen,et al.  Super‐resolution methods in MRI: Can they improve the trade‐off between resolution, signal‐to‐noise ratio, and acquisition time? , 2012, Magnetic resonance in medicine.

[19]  Hong Wang,et al.  Single image super-resolution via self-similarity and low-rank matrix recovery , 2017, Multimedia Tools and Applications.

[20]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Colin Studholme,et al.  A groupwise super-resolution approach: Application to brain MRI , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[22]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  François Rousseau,et al.  Brain Hallucination , 2008, ECCV.

[24]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Amod Jog,et al.  Improving magnetic resonance resolution with supervised learning , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[26]  Radu Timofte,et al.  2018 PIRM Challenge on Perceptual Image Super-resolution , 2018, ArXiv.

[27]  Shihui Ying,et al.  MR Image Super-Resolution via Wide Residual Networks With Fixed Skip Connection , 2019, IEEE Journal of Biomedical and Health Informatics.

[28]  Guang Yang,et al.  Combined self-learning based single-image super-resolution and dual-tree complex wavelet transform denoising for medical images , 2016, SPIE Medical Imaging.

[29]  Wiro J. Niessen,et al.  Super-Resolution Reconstruction Using Cross-Scale Self-similarity in Multi-slice MRI , 2013, MICCAI.

[30]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[31]  Mert R. Sabuncu,et al.  Medical Image Imputation From Image Collections , 2018, IEEE Transactions on Medical Imaging.

[32]  Borut Marincek,et al.  How Does MRI Work? An Introduction to the Physics and Function of Magnetic Resonance Imaging , 2007, Journal of Nuclear Medicine.

[33]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[34]  D. Louis Collins,et al.  Non-local MRI upsampling , 2010, Medical Image Anal..

[35]  F. Barkhof,et al.  The Holy Grail in diagnostic neuroradiology: 3T or 3D? , 2010, European Radiology.

[36]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[37]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[38]  Amod Jog,et al.  Self Super-Resolution for Magnetic Resonance Images , 2016, MICCAI.

[39]  Marios Politis,et al.  Advances in MRI Methodology. , 2018, International review of neurobiology.

[40]  Mert R. Sabuncu,et al.  Population Based Image Imputation , 2017, IPMI.

[41]  Hayit Greenspan,et al.  MRI Inter-slice Reconstruction Using Super-Resolution , 2001, MICCAI.

[42]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[43]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[44]  Damon M. Chandler,et al.  S3: A Spectral and Spatial Sharpness Measure , 2009, 2009 First International Conference on Advances in Multimedia.

[45]  D. Shen,et al.  LRTV: MR Image Super-Resolution With Low-Rank and Total Variation Regularizations , 2015, IEEE Transactions on Medical Imaging.