Frequency-Selective Learning for CT to MR Synthesis

Magnetic resonance (MR) and computed tomography (CT) images are important tools for brain studies, which noninvasively reveal the brain structure. However, the acquisition of MR images could be impractical under conditions where the imaging time is limited, and in many situations only CT images can be acquired. Although CT images provide valuable information about brain tissue, the anatomical structures are usually less distinguishable in CT than in MR images. To address this issue, convolutional neural networks (CNNs) have been developed to learn the mapping from CT to MR images, from which brains can be parcellated into anatomical regions for further analysis. However, it is observed that image synthesis based on CNNs tend to lose information about image details, which adversely affects the quality of the synthesized images. In this work, we propose frequency-selective learning for CT to MR image synthesis, where multiheads are used in the deep network for learning the mapping of different frequency components. The different frequency components are added to give the final output of the network. The network is trained by minimizing the weighted sum of the synthesis losses for the whole image and each frequency component. Experiments were performed on brain CT images, where the quality of the synthesized MR images was evaluated. Results show that the proposed method reduces the synthesis errors and improves the accuracy of the segmentation of brain structures based on the synthesized MR images.