Super-resolution reconstruction of knee magnetic resonance imaging based on deep learning

BACKGROUND AND OBJECTIVE With the rapid development of medical imaging and intelligent diagnosis, artificial intelligence methods have become a research hotspot of radiography processing technology in recent years. The low definition of knee magnetic resonance image texture seriously affects the diagnosis of knee osteoarthritis. This paper presents a super-resolution reconstruction method to address this problem. METHODS In this paper, we propose an efficient medical image super-resolution (EMISR) method, in which we mainly adopted three hidden layers of super-resolution convolution neural network (SRCNN) and a sub-pixel convolution layer of efficient sub-pixel convolution neural network (ESPCN). The addition of the efficient sub-pixel convolutional layer in the hidden layer and the small network replacement consisting of concatenated convolutions to address low-resolution images but not high-resolution images are important. The EMISR method also uses cascaded small convolution kernels to improve reconstruction speed and deepen the convolution neural network to improve reconstruction quality. RESULTS The proposed method is tested in the public dataset IDI, and the reconstruction quality of the algorithm is higher than that of the sparse coding-based network (SCN) method, the SRCNN method, and the ESPCN method (+ 2.306 dB, + 2.540 dB, + 1.089 dB improved); moreover, the reconstruction speed is faster than its counterparts (+ 4.272 s, + 1.967 s, and + 0.073 s improved). CONCLUSION The experimental results show that our EMISR framework has improved performance and greatly reduces the number of parameters and training time. Furthermore, the reconstructed image presents more details, and the edges are more complete. Therefore, the EMISR technique provides a more powerful medical analysis in knee osteoarthritis examinations.

[1]  Elena Deza,et al.  Encyclopedia of Distances , 2014 .

[2]  Zhang Jie Region-based Bicubic Image Interpolation Algorithm , 2010 .

[3]  Aamir Saeed Malik,et al.  Fusion of multinuclear magnetic resonance images of knee for the assessment of articular cartilage , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[4]  Wenhan Yang,et al.  Image Super-Resolution Based on Structure-Modulated Sparse Representation , 2015, IEEE Transactions on Image Processing.

[5]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[6]  Jae-Seok Choi,et al.  Single image super-resolution based on self-examples using context-dependent subpatches , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[7]  Aamir Saeed Malik,et al.  Non-invasive and in vivo assessment of osteoarthritic articular cartilage: a review on MRI investigations , 2014, Rheumatology International.

[8]  Changsheng Hu,et al.  Super-resolution of medical image using representation learning , 2016, 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP).

[9]  Dacheng Tao,et al.  Local Rademacher Complexity for Multi-Label Learning , 2014, IEEE Transactions on Image Processing.

[10]  Yuchen Yan,et al.  Adaptive shock filter for image super-resolution and enhancement , 2016, J. Vis. Commun. Image Represent..

[11]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Chih-Yuan Yang,et al.  Exploiting Self-similarities for Single Frame Super-Resolution , 2010, ACCV.

[13]  Volodymyr I. Ponomaryov,et al.  Super Resolution Imaging via Sparse Interpolation in Wavelet Domain with Implementation in DSP and GPU , 2014, CIARP.

[14]  Hamidreza Rashidy Kanan,et al.  Breast cancer detection and classification in digital mammography based on Non-Subsampled Contourlet Transform (NSCT) and Super Resolution , 2015, Comput. Methods Programs Biomed..

[15]  Rachid Deriche,et al.  The use of super‐resolution techniques to reduce slice thickness in functional MRI , 2004, Int. J. Imaging Syst. Technol..

[16]  Horst Bischof,et al.  Fast and accurate image upscaling with super-resolution forests , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Xueying Qin,et al.  Deep Depth Super-Resolution: Learning Depth Super-Resolution Using Deep Convolutional Neural Network , 2016, ACCV.

[18]  Lixin Zheng,et al.  A Fast Medical Image Super Resolution Method Based on Deep Learning Network , 2019, IEEE Access.

[19]  Anselmo Cardoso de Paiva,et al.  An automatic method for lung segmentation and reconstruction in chest X-ray using deep neural networks , 2019, Comput. Methods Programs Biomed..

[20]  Linhua Jiang,et al.  Super resolution reconstruction algorithm of video image based on deep self encoding learning , 2018, Multimedia Tools and Applications.

[21]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Dong-yuan Ge,et al.  Binocular vision calibration and 3D re-construction with an orthogonal learning neural network , 2016, Multimedia Tools and Applications.

[23]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..

[24]  Xiaoyi Feng,et al.  Single image super-resolution by combining self-learning and example-based learning methods , 2015, Multimedia Tools and Applications.

[25]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[26]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[27]  T. W. Anderson An Introduction to Multivariate Statistical Analysis , 1959 .

[28]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[29]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[30]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Lixin Zheng,et al.  A transfer learning method with deep residual network for pediatric pneumonia diagnosis , 2020, Comput. Methods Programs Biomed..

[32]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.