Self-supervised CT super-resolution with hybrid model

Software-based methods can improve CT spatial resolution without changing the hardware of the scanner or increasing the radiation dose to the object. In this work, we aim to develop a deep learning (DL) based CT super-resolution (SR) method that can reconstruct low-resolution (LR) sinograms into high-resolution (HR) CT images. We mathematically analyzed imaging processes in the CT SR imaging problem and synergistically integrated the SR model in the sinogram domain and the deblur model in the image domain into a hybrid model (SADIR). SADIR incorporates the CT domain knowledge and is unrolled into a DL network (SADIR-Net). The SADIR-Net is a self-supervised network, which can be trained and tested with a single sinogram. SADIR-Net was evaluated through SR CT imaging of a Catphan700 physical phantom and a real porcine phantom, and its performance was compared to the other state-of-the-art (SotA) DL-based CT SR methods. On both phantoms, SADIR-Net obtains the highest information fidelity criterion (IFC), structure similarity index (SSIM), and lowest root-mean-square-error (RMSE). As to the modulation transfer function (MTF), SADIR-Net also obtains the best result and improves the MTF50% by 69.2% and MTF10% by 69.5% compared with FBP. Alternatively, the spatial resolutions at MTF50% and MTF10% from SADIR-Net can reach 91.3% and 89.3% of the counterparts reconstructed from the HR sinogram with FBP. The results show that SADIR-Net can provide performance comparable to the other SotA methods for CT SR reconstruction, especially in the case of extremely limited training data or even no data at all. Thus, the SADIR method could find use in improving CT resolution without changing the hardware of the scanner or increasing the radiation dose to the object.

[1]  Hu Chen,et al.  LEARN: Learned Experts’ Assessment-Based Reconstruction Network for Sparse-Data CT , 2017, IEEE Transactions on Medical Imaging.

[2]  J. Fessler,et al.  Relaxed Linearized Algorithms for Faster X-Ray CT Image Reconstruction. , 2016, IEEE transactions on medical imaging.

[3]  Yaoqin Xie,et al.  A Sparse-View CT Reconstruction Method Based on Combination of DenseNet and Deconvolution , 2018, IEEE Transactions on Medical Imaging.

[4]  Zixiang Xiong,et al.  Dual-Path Deep Fusion Network for Face Image Hallucination , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Asif Mehmood,et al.  Multi-Step Reinforcement Learning for Single Image Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Xiyuan Hu,et al.  PCA-SRGAN: Incremental Orthogonal Projection Discrimination for Face Super-resolution , 2020, ACM Multimedia.

[7]  Liang Lin,et al.  Attention-Aware Face Hallucination via Deep Reinforcement Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Peyman Mostaghimi,et al.  Enhancing Resolution of Digital Rock Images with Super Resolution Convolutional Neural Networks , 2019, Journal of Petroleum Science and Engineering.

[9]  Hongming Shan,et al.  MRI Super-Resolution With Ensemble Learning and Complementary Priors , 2020, IEEE Transactions on Computational Imaging.

[10]  Yukai Wang,et al.  CT-image Super Resolution Using 3D Convolutional Neural Network , 2018, Comput. Geosci..

[11]  Norbert J. Pelc,et al.  Recent and Future Directions in CT Imaging , 2014, Annals of Biomedical Engineering.

[12]  L. Xing,et al.  Modularized Data-Driven Reconstruction Framework for Non-ideal Focal Spot Effect Elimination in Computed Tomography. , 2021, Medical physics.

[13]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[14]  Yue Zhou,et al.  Diagnostic value and key features of computed tomography in Coronavirus Disease 2019 , 2020, Emerging microbes & infections.

[15]  Wilfried Philips,et al.  Spatial-Spectral Structured Sparse Low-Rank Representation for Hyperspectral Image Super-Resolution , 2021, IEEE Transactions on Image Processing.

[16]  N. Saul,et al.  米国放射線科医学会(ACR)認定ファントムを用いてコンピュータ断層撮影(CT)変調用変成機能(MTF)およびノイズパワースペクトル(NPS)を測定するための単純なアプローチ , 2013 .

[17]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Fan Yang,et al.  PET image super-resolution using generative adversarial networks , 2020, Neural Networks.

[19]  Yaoqin Xie,et al.  A novel design of ultrafast micro-CT system based on carbon nanotube: A feasibility study in phantom. , 2016, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[20]  M. Jinzaki,et al.  Spatial resolution compensation by adjusting the reconstruction kernels for iterative reconstruction images of computed tomography. , 2020, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[21]  Junjun Jiang,et al.  Ensemble Super-Resolution With a Reference Dataset , 2020, IEEE Transactions on Cybernetics.

[22]  Michael M Lell,et al.  Recent and Upcoming Technological Developments in Computed Tomography: High Speed, Low Dose, Deep Learning, Multienergy. , 2019, Investigative radiology.

[23]  Amit Singhal,et al.  Modeling and prediction of COVID-19 pandemic using Gaussian mixture model , 2020, Chaos, Solitons & Fractals.

[24]  Lei Xing,et al.  TransCT: Dual-Path Transformer for Low Dose Computed Tomography , 2021, MICCAI.

[25]  Michal Irani,et al.  "Zero-Shot" Super-Resolution Using Deep Internal Learning , 2017, CVPR.

[26]  Junko Ota,et al.  Application of Super-Resolution Convolutional Neural Network for Enhancing Image Resolution in Chest CT , 2018, Journal of Digital Imaging.

[27]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  F. Beckmann,et al.  Ex vivo evaluation of an atherosclerotic human coronary artery via histology and high-resolution hard X-ray tomography , 2019, Scientific Reports.

[29]  S. Varadarajan,et al.  Deblurring techniques — A comprehensive survey , 2017, 2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI).

[30]  Yuanchun Zhou,et al.  Sparse representation based computed tomography images reconstruction by coupled dictionary learning algorithm , 2020, IET Image Process..

[31]  W Simon,et al.  Attenuation Correction in Gamma Emission Computed Tomography , 1981, Journal of computer assisted tomography.

[32]  Darryl McClymont,et al.  Improved compressed sensing and super‐resolution of cardiac diffusion MRI with structure‐guided total variation , 2020, Magnetic resonance in medicine.

[33]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[34]  Wei Zhao,et al.  Noise2Context: Context-assisted Learning 3D Thin-layer for Low Dose CT. , 2020, Medical physics.

[35]  Xinbo Gao,et al.  Single Image Super-Resolution via Multiple Mixture Prior Models , 2018, IEEE Transactions on Image Processing.

[36]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[37]  Menglei Zhang,et al.  Supervised Pixel-Wise GAN for Face Super-Resolution , 2021, IEEE Transactions on Multimedia.

[38]  Effect of Ultra High-Resolution Computed Tomography and Model-Based Iterative Reconstruction on Detectability of Simulated Submillimeter Artery. , 2020, Journal of computer assisted tomography.

[39]  Jinliang Zhang,et al.  A review of high-resolution X-ray computed tomography applied to petroleum geology and a case study. , 2019, Micron.

[40]  Mohammad Madallh Alhabahba,et al.  Contrasting Computational Models of Mate Preference Integration Across 45 Countries , 2019, Scientific Reports.

[41]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[42]  Lei Wang,et al.  A shallow convolutional neural network for blind image sharpness assessment , 2017, PloS one.

[43]  Shoujun Zhou,et al.  Incorporating the hybrid deformable model for improving the performance of abdominal CT segmentation via multi-scale feature fusion network , 2021, Medical Image Anal..

[44]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[45]  Jeffrey H Siewerdsen,et al.  A simple approach to measure computed tomography (CT) modulation transfer function (MTF) and noise-power spectrum (NPS) using the American College of Radiology (ACR) accreditation phantom. , 2013, Medical physics.

[46]  Lee W Goldman,et al.  Principles of CT: Radiation Dose and Image Quality* , 2007, Journal of Nuclear Medicine Technology.

[47]  Jeffrey A. Fessler,et al.  Image Reconstruction is a New Frontier of Machine Learning , 2018, IEEE Transactions on Medical Imaging.

[48]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  H. D. de Koning,et al.  Recommendations for Implementing Lung Cancer Screening with Low-Dose Computed Tomography in Europe , 2020, Cancers.

[50]  Y. Jung,et al.  Very deep super-resolution for efficient cone-beam computed tomographic image restoration , 2020, Imaging science in dentistry.

[51]  Guang Li,et al.  CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE) , 2018, IEEE Transactions on Medical Imaging.

[52]  Xun Jia,et al.  Physics Model-Based Scatter Correction in Multi-Source Interior Computed Tomography , 2018, IEEE Transactions on Medical Imaging.

[53]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[54]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[56]  Ke Gu,et al.  ATMFN: Adaptive-Threshold-Based Multi-Model Fusion Network for Compressed Face Hallucination , 2020, IEEE Transactions on Multimedia.

[57]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[58]  Lin Fu,et al.  Sinogram rebinning and frequency boosting for high resolution iterative CT reconstruction with focal spot deflection , 2014, Medical Imaging.