Super-resolution reconstruction of brain magnetic resonance images via lightweight autoencoder

Abstract Magnetic Resonance Imaging (MRI) is useful to provide detailed anatomical information such as images of tissues and organs within the body that are vital for quantitative image analysis. However, typically the MR images acquired lacks adequate resolution because of the constraints such as patients’ comfort and long sampling duration. Processing the low resolution MRI may lead to an incorrect diagnosis. Therefore, there is a need for super resolution techniques to obtain high resolution MRI images. Single image super resolution (SR) is one of the popular techniques to enhance image quality. Reconstruction based SR technique is a category of single image SR that can reconstruct the low resolution MRI images to high resolution images. Inspired by the advanced deep learning based SR techniques, in this paper we propose an autoencoder based MRI image super resolution technique that performs reconstruction of the high resolution MRI images from low resolution MRI images. Experimental results on synthetic and real brain MRI images show that our autoencoder based SR technique surpasses other state-of-the-art techniques in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), Information Fidelity Criterion (IFC), and computational time.

[1]  J. Karthikeyan,et al.  An Efficient Privacy-preserving Deep Learning Scheme for Medical Image Analysis , 2020 .

[2]  Zhou Wang,et al.  Structural Approaches to Image Quality Assessment , 2005 .

[3]  Zhe Gan,et al.  Variational Autoencoder for Deep Learning of Images, Labels and Captions , 2016, NIPS.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[7]  J. Andrew,et al.  Spine Magnetic Resonance Image Segmentation Using Deep Learning Techniques , 2020, 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS).

[8]  Chih-Yuan Yang,et al.  Fast Direct Super-Resolution by Simple Functions , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Feng Shi,et al.  Brain MRI super resolution using 3D deep densely connected neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[10]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[11]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[12]  D. Louis Collins,et al.  Non-local MRI upsampling , 2010, Medical Image Anal..

[13]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[14]  R. Manicka Chezian,et al.  Edge Detection Operators: Peak Signal to Noise Ratio Based Comparison , 2014 .

[15]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Jie Li,et al.  Progressive Sub-Band Residual-Learning Network for MR Image Super Resolution , 2020, IEEE Journal of Biomedical and Health Informatics.

[18]  Ge Wang,et al.  Super-resolution MRI through Deep Learning , 2018, 1810.06776.

[19]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[20]  Michael T. Orchard,et al.  Novel sequential error-concealment techniques using orientation adaptive interpolation , 2001, IEEE Trans. Circuits Syst. Video Technol..

[21]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[22]  Dinggang Shen,et al.  Convolutional Neural Network for Reconstruction of 7T-like Images from 3T MRI Using Appearance and Anatomical Features , 2016, LABELS/DLMIA@MICCAI.

[23]  Tao Zhang,et al.  Channel Splitting Network for Single MR Image Super-Resolution , 2018, IEEE Transactions on Image Processing.

[24]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[26]  Jaime Lloret,et al.  Conditional Variational Autoencoder for Prediction and Feature Recovery Applied to Intrusion Detection in IoT , 2017, Sensors.

[27]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Eser Sert,et al.  An expert system for brain tumor detection: Fuzzy C-means with super resolution and convolutional neural network with extreme learning machine. , 2019, Medical hypotheses.

[32]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[33]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Lei Wang,et al.  Remote Sensing Image Super-Resolution Using Sparse Representation and Coupled Sparse Autoencoder , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Hongming Shan,et al.  MRI Super-Resolution With Ensemble Learning and Complementary Priors , 2020, IEEE Transactions on Computational Imaging.

[36]  Sébastien Ourselin,et al.  An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain MRI , 2019, NeuroImage.

[37]  Yu Yang,et al.  Simultaneous single- and multi-contrast super-resolution for brain MRI images based on a convolutional neural network , 2018, Comput. Biol. Medicine.

[38]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[39]  Yizhen Huang,et al.  Super-resolution using neural networks based on the optimal recovery theory , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[40]  Hiromitsu Furukawa,et al.  Super-resolution technique for high-resolution multichannel Fourier transform spectrometer. , 2018, Optics express.

[41]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[42]  Martin O. Leach,et al.  Super-resolution T2-weighted 4D MRI for image guided radiotherapy , 2018, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[43]  Jae Sung Lee,et al.  Computed tomography super-resolution using deep convolutional neural network , 2018, Physics in medicine and biology.

[44]  H. Gach,et al.  Autoencoder-Inspired Convolutional Network-Based Super-Resolution Method in MRI , 2021, IEEE Journal of Translational Engineering in Health and Medicine.

[45]  Thomas S. Huang,et al.  Image Super-Resolution via Dual-State Recurrent Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[46]  Rex Fiona,et al.  Comparative Study of Various Deep Convolutional Neural Networks in the Early Prediction of Cancer , 2019, 2019 International Conference on Intelligent Computing and Control Systems (ICCS).

[47]  D. Tank,et al.  Brain magnetic resonance imaging with contrast dependent on blood oxygenation. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Mohanasankar Sivaprakasam,et al.  MRI Super-Resolution using Laplacian Pyramid Convolutional Neural Networks with Isotropic Undecimated Wavelet Loss , 2020, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[49]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[50]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[51]  Yuanyuan Jia,et al.  Brain MRI Super-Resolution Using 3D Dilated Convolutional Encoder–Decoder Network , 2020, IEEE Access.

[52]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[53]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[54]  Nathan Lay,et al.  Automatic magnetic resonance prostate segmentation by deep learning with holistically nested networks , 2017, Journal of medical imaging.

[55]  Vanita Mane,et al.  Image Super-Resolution for MRI Images using 3D Faster Super-Resolution Convolutional Neural Network architecture , 2020, ITM Web of Conferences.

[56]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[57]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[58]  Fei Zhou,et al.  Interpolation-Based Image Super-Resolution Using Multisurface Fitting , 2012, IEEE Transactions on Image Processing.

[59]  Nicolas Passat,et al.  Multiscale brain MRI super-resolution using deep 3D convolutional networks , 2019, Comput. Medical Imaging Graph..

[60]  D. Shen,et al.  LRTV: MR Image Super-Resolution With Low-Rank and Total Variation Regularizations , 2015, IEEE Transactions on Medical Imaging.

[61]  Tieniu Tan,et al.  Wavelet-SRNet: A Wavelet-Based CNN for Multi-scale Face Super Resolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[62]  Xuelong Li,et al.  Single-image super-resolution via local learning , 2011, Int. J. Mach. Learn. Cybern..

[63]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).