Channel Splitting Network for Single MR Image Super-Resolution

High resolution magnetic resonance (MR) imaging is desirable in many clinical applications due to its contribution to more accurate subsequent analyses and early clinical diagnoses. Single image super-resolution (SISR) is an effective and cost efficient alternative technique to improve the spatial resolution of MR images. In the past few years, SISR methods based on deep learning techniques, especially convolutional neural networks (CNNs), have achieved the state-of-the-art performance on natural images. However, the information is gradually weakened and training becomes increasingly difficult as the network deepens. The problem is more serious for medical images because lacking high quality and effective training samples makes deep models prone to underfitting or overfitting. Nevertheless, many current models treat the hierarchical features on different channels equivalently, which is not helpful for the models to deal with the hierarchical features discriminatively and targetedly. To this end, we present a novel channel splitting network (CSN) to ease the representational burden of deep models. The proposed CSN model divides the hierarchical features into two branches, i.e., residual branch and dense branch, with different information transmissions. The residual branch is able to promote feature reuse, while the dense branch is beneficial to the exploration of new features. Besides, we also adopt the merge-and-run mapping to facilitate information integration between different branches. The extensive experiments on various MR images, including proton density (PD), T1, and T2 images, show that the proposed CSN model achieves superior performance over other state-of-the-art SISR methods.

[1]  Konstantinos Kamnitsas,et al.  Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks , 2016, MICCAI.

[2]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Shihui Ying,et al.  Super-resolution reconstruction of MR image with a novel residual learning network algorithm , 2018, Physics in medicine and biology.

[4]  Aaron Carass,et al.  Self super-resolution for magnetic resonance images using deep networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[5]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[6]  Xi Wu,et al.  Single image super resolution of 3D MRI using local regression and intermodality priors , 2016, International Conference on Digital Image Processing.

[7]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  H Stark,et al.  High-resolution image recovery from image-plane arrays, using convex projections. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[10]  D. Louis Collins,et al.  MRI Superresolution Using Self-Similarity and Image Priors , 2010, Int. J. Biomed. Imaging.

[11]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[12]  Daniel Rueckert,et al.  Cardiac Image Super-Resolution with Global Correspondence Using Multi-Atlas PatchMatch , 2013, MICCAI.

[13]  Jungwon Lee,et al.  Image Super Resolution Based on Fusing Multiple Convolution Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[15]  Christopher Kanan,et al.  Aerial Spectral Super-Resolution using Conditional Adversarial Networks , 2017, ArXiv.

[16]  S Peled,et al.  Superresolution in MRI: Application to human white matter fiber tract visualization by diffusion tensor imaging , 2001, Magnetic resonance in medicine.

[17]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Hongyu Wang,et al.  End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks , 2016, IEEE Access.

[19]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Debiao Li,et al.  Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network , 2018, MICCAI.

[22]  Chi-Hieu Pham,et al.  Brain MRI super-resolution using deep 3D convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[23]  Thomas S. Huang,et al.  Coupled Dictionary Training for Image Super-Resolution , 2012, IEEE Transactions on Image Processing.

[24]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Zongxu Pan,et al.  Super-Resolution Based on Compressive Sensing and Structural Self-Similarity for Remote Sensing Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[26]  John F. Roddick,et al.  Sparse representation-based MRI super-resolution reconstruction , 2014 .

[27]  Takio Kurita,et al.  Fast and Accurate Image Super Resolution by Deep CNN with Skip Connection and Network in Network , 2017, ICONIP.

[28]  Russell M. Mersereau,et al.  A Super-Resolution Framework for 3-D High-Resolution and High-Contrast Imaging Using 2-D Multislice MRI , 2009, IEEE Transactions on Medical Imaging.

[29]  Chi-Wang Shu,et al.  On the Gibbs Phenomenon and Its Resolution , 1997, SIAM Rev..

[30]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Hayit Greenspan,et al.  MRI Inter-slice Reconstruction Using Super-Resolution , 2001, MICCAI.

[33]  Shihui Ying,et al.  MR Image Super-Resolution via Wide Residual Networks With Fixed Skip Connection , 2019, IEEE Journal of Biomedical and Health Informatics.

[34]  Jong Beom Ra,et al.  Example-Based Super-Resolution via Structure Analysis of Patches , 2013, IEEE Signal Processing Letters.

[35]  Jie Li,et al.  Single Image Super-Resolution via Cascaded Multi-Scale Cross Network , 2018, ArXiv.

[36]  Shuicheng Yan,et al.  Dual Path Networks , 2017, NIPS.

[37]  Feng Shi,et al.  Brain MRI super resolution using 3D deep densely connected neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[38]  Toufeeq Ahmad,et al.  An Integrated Interpolation-based Super Resolution Reconstruction Algorithm for Video Surveillance , 2012, J. Commun..

[39]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[41]  D. Louis Collins,et al.  Non-local MRI upsampling , 2010, Medical Image Anal..

[42]  M. Rahmati,et al.  Super-resolution MRI images using Compressive Sensing , 2012, 20th Iranian Conference on Electrical Engineering (ICEE2012).

[43]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[44]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Yueting Zhuang,et al.  Deep Convolutional Neural Networks with Merge-and-Run Mappings , 2016, IJCAI.

[46]  N. Alon,et al.  Resolution enhancement in MRI. , 2006, Magnetic resonance imaging.

[47]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[48]  Yizhen Huang,et al.  Super-resolution using neural networks based on the optimal recovery theory , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[49]  Peter M. Atkinson,et al.  Sub‐pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super‐resolution pixel‐swapping , 2006 .

[50]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[52]  François Rousseau,et al.  Brain Hallucination , 2008, ECCV.

[53]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[54]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[55]  Norberto Malpica,et al.  Single-image super-resolution of brain MR images using overcomplete dictionaries , 2013, Medical Image Anal..

[56]  Wiro J Niessen,et al.  Super‐resolution methods in MRI: Can they improve the trade‐off between resolution, signal‐to‐noise ratio, and acquisition time? , 2012, Magnetic resonance in medicine.