Single MR Image Super-Resolution via Channel Splitting and Serial Fusion Network

Spatial resolution is a critical imaging parameter in magnetic resonance imaging (MRI). Acquiring high resolution MRI data usually takes long scanning time and would subject to motion artifacts due to hardware, physical, and physiological limitations. Single image super-resolution (SISR), especially that based on deep learning techniques, is an effective and promising alternative technique to improve the current spatial resolution of magnetic resonance (MR) images. However, the deeper network is more difficult to be effectively trained because the information is gradually weakened as the network deepens. This problem becomes more serious for medical images due to the degradation of training examples. In this paper, we present a novel channel splitting and serial fusion network (CSSFN) for single MR image super-resolution. Specifically, the proposed CSSFN network splits the hierarchical features into a series of subfeatures, which are then integrated together in a serial manner. Thus, the network becomes deeper and can deal with the subfeatures on different channels discriminatively. Besides, a dense global feature fusion (DGFF) is adopted to integrate the intermediate features, which further promotes the information flow in the network. Extensive experiments on several typical MR images show the superiority of our CSSFN model over other advanced SISR methods.

[1]  John F. Roddick,et al.  Sparse representation-based MRI super-resolution reconstruction , 2014 .

[2]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[4]  Yucheng Wang,et al.  Deep Bi-Dense Networks for Image Super-Resolution , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[5]  S Peled,et al.  Superresolution in MRI: Application to human white matter fiber tract visualization by diffusion tensor imaging , 2001, Magnetic resonance in medicine.

[6]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Tat-Seng Chua,et al.  SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ruslan Salakhutdinov,et al.  Generating Images from Captions with Attention , 2015, ICLR.

[9]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[10]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  François Rousseau,et al.  Brain Hallucination , 2008, ECCV.

[12]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yueting Zhuang,et al.  Deep Convolutional Neural Networks with Merge-and-Run Mappings , 2016, IJCAI.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Debiao Li,et al.  Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network , 2018, MICCAI.

[18]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[19]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  M. Rahmati,et al.  Super-resolution MRI images using Compressive Sensing , 2012, 20th Iranian Conference on Electrical Engineering (ICEE2012).

[21]  Jie Li,et al.  Channel-Wise and Spatial Feature Modulation Network for Single Image Super-Resolution , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[23]  Tao Zhang,et al.  Channel Splitting Network for Single MR Image Super-Resolution , 2018, IEEE Transactions on Image Processing.

[24]  N. Alon,et al.  Resolution enhancement in MRI. , 2006, Magnetic resonance imaging.

[25]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[26]  Xi Wu,et al.  Single image super resolution of 3D MRI using local regression and intermodality priors , 2016, International Conference on Digital Image Processing.

[27]  Feng Shi,et al.  Brain MRI super resolution using 3D deep densely connected neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[28]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[29]  Guang Li,et al.  CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE) , 2018, IEEE Transactions on Medical Imaging.

[30]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  H Stark,et al.  High-resolution image recovery from image-plane arrays, using convex projections. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[34]  Thomas S. Huang,et al.  Coupled Dictionary Training for Image Super-Resolution , 2012, IEEE Transactions on Image Processing.

[35]  Konstantinos Kamnitsas,et al.  Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks , 2016, MICCAI.

[36]  Chi-Wang Shu,et al.  On the Gibbs Phenomenon and Its Resolution , 1997, SIAM Rev..

[37]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .

[38]  Hayit Greenspan,et al.  MRI inter-slice reconstruction using super-resolution , 2002 .

[39]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[40]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Norberto Malpica,et al.  Single-image super-resolution of brain MR images using overcomplete dictionaries , 2013, Medical Image Anal..

[42]  Jong Beom Ra,et al.  Example-Based Super-Resolution via Structure Analysis of Patches , 2013, IEEE Signal Processing Letters.

[43]  Jie Li,et al.  Single Image Super-Resolution via Cascaded Multi-Scale Cross Network , 2018, ArXiv.

[44]  Chi-Hieu Pham,et al.  Brain MRI super-resolution using deep 3D convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[45]  Wiro J Niessen,et al.  Super‐resolution methods in MRI: Can they improve the trade‐off between resolution, signal‐to‐noise ratio, and acquisition time? , 2012, Magnetic resonance in medicine.

[46]  Russell M. Mersereau,et al.  A Super-Resolution Framework for 3-D High-Resolution and High-Contrast Imaging Using 2-D Multislice MRI , 2009, IEEE Transactions on Medical Imaging.

[47]  D. Louis Collins,et al.  MRI Superresolution Using Self-Similarity and Image Priors , 2010, Int. J. Biomed. Imaging.

[48]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Truong Q. Nguyen,et al.  Novel Example-Based Method for Super-Resolution and Denoising of Medical Images , 2014, IEEE Transactions on Image Processing.

[50]  Ying Tai,et al.  SESR: Single Image Super Resolution with Recursive Squeeze and Excitation Networks , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[51]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Wei Wang,et al.  Deep Learning for Single Image Super-Resolution: A Brief Review , 2018, IEEE Transactions on Multimedia.

[53]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Aaron Carass,et al.  Self super-resolution for magnetic resonance images using deep networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[55]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[56]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[59]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[60]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).