Two-Stage Self-Supervised Cycle-Consistency Network for Reconstruction of Thin-Slice MR Images

The thick-slice magnetic resonance (MR) images are often structurally blurred in coronal and sagittal views, which causes harm to diagnosis and image post-processing. Deep learning (DL) has shown great potential to reconstruct the high-resolution (HR) thin-slice MR images from those low-resolution (LR) cases, which we refer to as the slice interpolation task in this work. However, since it is generally difficult to sample abundant paired LR-HR MR images, the classical fully supervised DL-based models cannot be effectively trained to get robust performance. To this end, we propose a novel Two-stage Self-supervised Cycle-consistency Network (TSCNet) for MR slice interpolation, in which a twostage self-supervised learning (SSL) strategy is developed for unsupervised DL network training. The paired LR-HR images are synthesized along the sagittal and coronal directions of input LR images for network pretraining in the firststage SSL, and then a cyclic interpolation procedure based on triplet axial slices is designed in the second-stage SSL for further refinement. More training samples with rich contexts along all directions are exploited as guidance to guarantee the improved interpolation performance. Moreover, a new cycle-consistency constraint is proposed to supervise this cyclic procedure, which encourages the network to reconstruct more realistic HR images. The experimental results on a real MRI dataset indicate that TSCNet achieves superior performance over the conventional and other SSL-based algorithms, and obtains competitive qualitative and quantitative results compared with the fully supervised algorithm.

[1]  Rama Chellappa,et al.  SAINT: Spatially Aware Interpolation NeTwork for Medical Slice Synthesis , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jinhua Yu,et al.  Reconstruction of Thin-Slice Medical Images Using Generative Adversarial Network , 2017, MLMI@MICCAI.

[3]  Simon K. Warfield,et al.  A New Sparse Representation Framework for Reconstruction of an Isotropic High Spatial Resolution MR Volume From Orthogonal Anisotropic Resolution Scans , 2017, IEEE Transactions on Medical Imaging.

[4]  Wei Wang,et al.  Deep Learning for Single Image Super-Resolution: A Brief Review , 2018, IEEE Transactions on Multimedia.

[5]  Jerry L Prince,et al.  SMORE: A Self-Supervised Anti-Aliasing and Super-Resolution Algorithm for MRI Using Deep Learning , 2020, IEEE Transactions on Medical Imaging.

[6]  Aaron Carass,et al.  Self super-resolution for magnetic resonance images using deep networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[7]  Eric Van Reeth,et al.  Super-resolution in magnetic resonance imaging: A review , 2012 .

[8]  Siyuan Liu,et al.  Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Yiran Li,et al.  A Two-Stage Multi-loss Super-Resolution Network for Arterial Spin Labeling Magnetic Resonance Imaging , 2019, MICCAI.

[10]  Shihui Ying,et al.  MR Image Super-Resolution via Wide Residual Networks With Fixed Skip Connection , 2019, IEEE Journal of Biomedical and Health Informatics.

[11]  Yingli Tian,et al.  Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Feng Shi,et al.  Brain MRI super resolution using 3D deep densely connected neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[13]  Zhongshi He,et al.  Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network , 2020, Neurocomputing.

[14]  Wiro J Niessen,et al.  Super‐resolution methods in MRI: Can they improve the trade‐off between resolution, signal‐to‐noise ratio, and acquisition time? , 2012, Magnetic resonance in medicine.

[15]  Nick C Fox,et al.  The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods , 2008, Journal of magnetic resonance imaging : JMRI.

[16]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jan Kautz,et al.  Unsupervised Video Interpolation Using Cycle Consistency , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Peter A. Calabresi,et al.  A Deep Learning Based Anti-aliasing Self Super-Resolution Algorithm for MRI , 2018, MICCAI.

[19]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.