Super-resolution of Pneumocystis carinii pneumonia CT via self-attention GAN

BACKGROUND AND OBJECTIVE Computed tomography (CT) examination plays an important role in screening suspected and confirmed patients in pneumocystis carinii pneumonia (PCP), and the efficient acquisition of high-quality medical CT images is essential for the clinical application of computer-aided diagnosis technology. Therefore, improving the resolution of CT images of pneumonia is a very important task. METHODS Aiming at the problem of how to recover the texture details of the reconstructed PCP CT super-resolution image, we propose the image super-resolution reconstruction model based on self-attention generation adversarial network (SAGAN). In the SAGAN algorithm, a generator based on self-attention mechanism and residual module is used to transform a low-resolution image into a super-resolution image. A discriminator based on depth convolution network tries to distinguish the difference between the reconstructed super-resolution image and the real super-resolution image. In terms of loss function construction, on the one hand, the Charbonnier content loss function is used to improve the accuracy of image reconstruction, and on the other hand, the feature value before activation of the pre-trained VGGNet is used to calculate the perceptual loss to achieve accurate texture detail reconstruction of super-resolution images. RESULTS Experimental results show that our SAGAN algorithm is superior to other state-of-the-art algorithms in both peak signal-to-noise ratio (PSNR) and structural similarity score (SSIM). Specifically, our SAGAN method can obtain 31.94 dB which is 1.53 dB better than SRGAN on Set5 dataset for 4 enlargements. CONCLUSION Our SAGAN method can reconstruct more realistic PCP CT images with clear texture, which can help experts diagnose the condition of PCP.

[1]  Tinghui Ouyang,et al.  Two-phase deep learning model for short-term wind direction forecasting , 2021 .

[2]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  K. V. Arya,et al.  A robust face super-resolution algorithm and its application in low-resolution face recognition system , 2020, Multimedia Tools and Applications.

[4]  Peter M. Atkinson,et al.  Sub‐pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super‐resolution pixel‐swapping , 2006 .

[5]  Yuhu Cheng,et al.  Multi-window back-projection residual networks for reconstructing COVID-19 CT super-resolution images , 2021, Computer Methods and Programs in Biomedicine.

[6]  K. Norris,et al.  Pneumocystis Pneumonia: Immunity, Vaccines, and Treatments , 2021, Pathogens.

[7]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  D. Perl,et al.  Pneumocystis carinii pneumonia in the United States: epidemiologic, diagnostic, and clinical features. , 1976, National Cancer Institute monograph.

[10]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[11]  Lixin Zheng,et al.  Multiple improved residual networks for medical image super-resolution , 2021, Future Gener. Comput. Syst..

[12]  J. Shelhamer,et al.  Pneumocystis carinii pneumonia: a comparison between patients with the acquired immunodeficiency syndrome and patients with other immunodeficiencies. , 1984, Annals of internal medicine.

[13]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[16]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[18]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Wangmeng Zuo,et al.  Learning a Single Convolutional Super-Resolution Network for Multiple Degradations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jun Liang,et al.  Radiation pneumonitis complicated by Pneumocystis carinii in patients with thoracic neoplasia: a clinical analysis of 7 cases , 2019, Cancer communications.

[22]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[23]  Jianqing Zhu,et al.  Super-resolution reconstruction of knee magnetic resonance imaging based on deep learning , 2020, Comput. Methods Programs Biomed..

[24]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  L. Dunphy,et al.  Missed opportunity to diagnose HIV with Pneumocystis carinii pneumonia as its sequela , 2020, BMJ Case Reports.

[27]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .