Synthesized 7T MRI from 3T MRI via deep learning in spatial and wavelet domains

Ultra-high field 7T MRI scanners, while producing images with exceptional anatomical details, are cost prohibitive and hence highly inaccessible. In this paper, we introduce a novel deep learning network that fuses complementary information from spatial and wavelet domains to synthesize 7T T1-weighted images from their 3T counterparts. Our deep learning network leverages wavelet transformation to facilitate effective multi-scale reconstruction, taking into account both low-frequency tissue contrast and high-frequency anatomical details. Our network utilizes a novel wavelet-based affine transformation (WAT) layer, which modulates feature maps from the spatial domain with information from the wavelet domain. Extensive experimental results demonstrate the capability of the proposed method in synthesizing high-quality 7T images with better tissue contrast and greater details, outperforming state-of-the-art methods.

[1]  Dinggang Shen,et al.  Neonatal brain image segmentation in longitudinal MRI studies , 2010, NeuroImage.

[2]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Qian Wang,et al.  Deep embedding convolutional neural network for synthesizing CT image from T1‐Weighted MR image , 2018, Medical Image Anal..

[4]  Yaozong Gao,et al.  CT male pelvic organ segmentation using fully convolutional networks with boundary sensitive representation , 2019, Medical Image Anal..

[5]  Narendra Ahuja,et al.  Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Dinggang Shen,et al.  Hierarchical Patch-Based Sparse Representation—A New Approach for Resolution Enhancement of 4D-CT Lung Data , 2012, IEEE Transactions on Medical Imaging.

[7]  Norberto Malpica,et al.  Single-image super-resolution of brain MR images using overcomplete dictionaries , 2013, Medical Image Anal..

[8]  Dinggang Shen,et al.  Convolutional Neural Network for Reconstruction of 7T-like Images from 3T MRI Using Appearance and Anatomical Features , 2016, LABELS/DLMIA@MICCAI.

[9]  Alan C. Evans,et al.  A nonparametric method for automatic correction of intensity nonuniformity in MRI data , 1998, IEEE Transactions on Medical Imaging.

[10]  Dinggang Shen,et al.  Multi‐channel multi‐scale fully convolutional network for 3D perivascular spaces segmentation in 7T MR images , 2018, Medical Image Anal..

[11]  R. Haddad,et al.  Multiresolution Signal Decomposition: Transforms, Subbands, and Wavelets , 1992 .

[12]  Yaozong Gao,et al.  7T‐guided super‐resolution of 3T MRI , 2017, Medical physics.

[13]  Colin Studholme,et al.  Intersection Based Motion Correction of Multislice MRI for 3-D in Utero Fetal Brain Image Formation , 2010, IEEE Transactions on Medical Imaging.

[14]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[15]  Konstantinos Kamnitsas,et al.  Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks , 2016, MICCAI.

[16]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[17]  D. Louis Collins,et al.  MRI Superresolution Using Self-Similarity and Image Priors , 2010, Int. J. Biomed. Imaging.

[18]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Antonio Criminisi,et al.  Image Quality Transfer via Random Forest Regression: Applications in Diffusion MRI , 2014, MICCAI.

[20]  Brian B. Avants,et al.  An Open Source Multivariate Framework for n-Tissue Segmentation with Evaluation on Public Data , 2011, Neuroinformatics.

[21]  Dinggang Shen,et al.  Hierarchical Reconstruction of 7T-like Images from 3T MRI Using Multi-level CCA and Group Sparsity , 2015, MICCAI.

[22]  Wan-Chi Siu,et al.  A Modified Edge Directed Interpolation for images , 2009, 2009 17th European Signal Processing Conference.

[23]  B. Forstmann,et al.  Ultra High Field MRI-Guided Deep Brain Stimulation. , 2017, Trends in biotechnology.

[24]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[25]  Alan C. Evans,et al.  Enhancement of MR Images Using Registration for Signal Averaging , 1998, Journal of Computer Assisted Tomography.

[26]  J. Zwanenburg,et al.  Clinical applications of 7 T MRI in the brain. , 2013, European journal of radiology.

[27]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[28]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Xinbo Gao,et al.  Fast and Accurate Single Image Super-Resolution via Information Distillation Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[32]  D. Shen,et al.  LRTV: MR Image Super-Resolution With Low-Rank and Total Variation Regularizations , 2015, IEEE Transactions on Medical Imaging.

[33]  Tieniu Tan,et al.  Wavelet-SRNet: A Wavelet-Based CNN for Multi-scale Face Super Resolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  Dinggang Shen,et al.  Dual-Domain Cascaded Regression for Synthesizing 7T from 3T MRI , 2018, MICCAI.

[35]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Chih-Yuan Yang,et al.  Single-Image Super-Resolution: A Benchmark , 2014, ECCV.

[37]  Stephen M. Smith,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[38]  Ingrid Daubechies,et al.  The wavelet transform, time-frequency localization and signal analysis , 1990, IEEE Trans. Inf. Theory.

[39]  Shuai Wang,et al.  Wavelet-based Semi-supervised Adversarial Learning for Synthesizing Realistic 7T from 3T MRI , 2019, MICCAI.