Promising Generative Adversarial Network Based Sinogram Inpainting Method for Ultra-Limited-Angle Computed Tomography Imaging

Limited-angle computed tomography (CT) image reconstruction is a challenging problem in the field of CT imaging. In some special applications, limited by the geometric space and mechanical structure of the imaging system, projections can only be collected with a scanning range of less than 90°. We call this kind of serious limited-angle problem the ultra-limited-angle problem, which is difficult to effectively alleviate by traditional iterative reconstruction algorithms. With the development of deep learning, the generative adversarial network (GAN) performs well in image inpainting tasks and can add effective image information to restore missing parts of an image. In this study, given the characteristic of GAN to generate missing information, the sinogram-inpainting-GAN (SI-GAN) is proposed to restore missing sinogram data to suppress the singularity of the truncated sinogram for ultra-limited-angle reconstruction. We propose the U-Net generator and patch-design discriminator in SI-GAN to make the network suitable for standard medical CT images. Furthermore, we propose a joint projection domain and image domain loss function, in which the weighted image domain loss can be added by the back-projection operation. Then, by inputting a paired limited-angle/180° sinogram into the network for training, we can obtain the trained model, which has extracted the continuity feature of sinogram data. Finally, the classic CT reconstruction method is used to reconstruct the images after obtaining the estimated sinograms. The simulation studies and actual data experiments indicate that the proposed method performed well to reduce the serious artifacts caused by ultra-limited-angle scanning.

[1]  李磊,et al.  Image reconstruction based on total-variation minimization and alternating direction method in linear scan computed tomography , 2013 .

[2]  F. Natterer.  The Mathematics of Computerized Tomography , 1986 .

[3]  Olaf Ronneberger,et al.  Invited Talk: U-Net Convolutional Networks for Biomedical Image Segmentation , 2017, Bildverarbeitung für die Medizin.

[4]  Swetava Ganguli,et al.  GeoGAN: A Conditional GAN with Reconstruction and Style Loss to Generate Standard Layer of Maps from Satellite Images , 2019, ArXiv.

[5]  H. Tuy.  AN INVERSION FORMULA FOR CONE-BEAM RECONSTRUCTION* , 1983 .

[6]  E. Sidky,et al.  Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT , 2009, 0904.4495.

[7]  Tzu-Chien Hsiao,et al.  Development of Limited-Angle Iterative Reconstruction Algorithms with Context Encoder-Based Sinogram Completion for Micro-CT Applications , 2018, Sensors.

[8]  Jongha Lee,et al.  View-interpolation of sparsely sampled sinogram using convolutional neural network , 2017, Medical Imaging.

[9]  Frank Natterer,et al.  Mathematical methods in image reconstruction , 2001, SIAM monographs on mathematical modeling and computation.

[10]  Hengyong Yu,et al.  Compressed sensing based interior tomography , 2009, Physics in medicine and biology.

[11]  Chuan Li,et al.  Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Mannudeep K. Kalra,et al.  Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN) , 2017, ArXiv.

[13]  Bin Dong,et al.  End-to-End Abnormality Detection in Medical Imaging , 2018, ArXiv.

[14]  R. Siddon.  Fast calculation of the exact radiological path for a three-dimensional CT array. , 1985, Medical physics.

[15]  Alexander Katsevich,et al.  Theoretically Exact Filtered Backprojection-Type Inversion Algorithm for Spiral CT , 2002, SIAM J. Appl. Math..

[16]  Jong Chul Ye,et al.  Multi-Scale Wavelet Domain Residual Learning for Limited-Angle CT Reconstruction , 2017, ArXiv.

[17]  Bruce D. Smith.  Image Reconstruction from Cone-Beam Projections: Necessary and Sufficient Conditions and Reconstruction Methods , 1985, IEEE Transactions on Medical Imaging.

[18]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Li Zhang,et al.  Direct filtered-backprojection-type reconstruction from a straight-line trajectory , 2007 .

[20]  Tamon Inouye,et al.  Image Reconstruction with Limited Angle Projection Data , 1979, IEEE Transactions on Nuclear Science.

[21]  Chuan Li,et al.  Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks , 2016, ECCV.

[22]  Zhiqiang Chen,et al.  Unsupervised Learnable Sinogram Inpainting Network (SIN) for Limited Angle CT reconstruction. , 2018, 1811.03911.

[23]  Jong Chul Ye,et al.  Deep Learning Reconstruction for 9-View Dual Energy CT Baggage Scanner , 2018, ArXiv.

[24]  A. H. Andersen.  Algebraic reconstruction in CT from limited views. , 1989, IEEE transactions on medical imaging.

[25]  Xiao Han,et al.  Optimization-based reconstruction of sparse images from few-view projections , 2012, Physics in medicine and biology.

[26]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[27]  Yuxiang Xing,et al.  Improve angular resolution for sparse-view CT with residual convolutional neural network , 2018, Medical Imaging.

[28]  Lei Li,et al.  A promising limited angular computed tomography reconstruction via segmentation based regional enhancement and total variation minimization. , 2016, The Review of scientific instruments.

[29]  Matti Lassas,et al.  Wavelet-based reconstruction for limited-angle X-ray tomography , 2006, IEEE Transactions on Medical Imaging.

[30]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[31]  Ge Wang,et al.  A Perspective on Deep Imaging , 2016, IEEE Access.

[32]  M. Kachelriess,et al.  Improved total variation-based CT image reconstruction applied to clinical data , 2011, Physics in medicine and biology.

[33]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[34]  Martin Benning,et al.  Directional sinogram inpainting for limited angle tomography , 2018, Inverse Problems.

[35]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[36]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Bin Yan,et al.  A sinogram inpainting method based on generative adversarial network for limited-angle computed tomography , 2019, 15th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine.

[38]  E. Sidky,et al.  Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization , 2008, Physics in medicine and biology.

[39]  Icru.,et al.  Phantoms and Computational Models in Therapy, Diagnosis and Protection , 1992 .

[40]  Bin Yan,et al.  Image Prediction for Limited-angle Tomography via Deep Learning with Convolutional Neural Network , 2016, ArXiv.

[41]  Lubomir M. Hadjiiski,et al.  A comparative study of limited-angle cone-beam reconstruction methods for breast tomosynthesis. , 2006, Medical physics.

[42]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[43]  Bin Yan,et al.  Edge guided image reconstruction in linear scan CT by weighted alternating direction TV minimization. , 2014, Journal of X-ray science and technology.

[44]  Jie Tang,et al.  Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets. , 2008, Medical physics.

[45]  Jan Sijbers,et al.  The ASTRA Toolbox: A platform for advanced algorithm development in electron tomography. , 2015, Ultramicroscopy.

[46]  Aykut Erdem,et al.  Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks , 2018, IEEE Transactions on Medical Imaging.

[47]  V. Perez-Mendez,et al.  TOMOGRAPHICAL IMAGING WITH LIMITED‐ANGLE INPUT , 1982 .

[48]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[49]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[50]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[51]  Miao Zhang,et al.  Ancient Painting to Natural Image: A New Solution for Painting Processing , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[52]  Jian Sun,et al.  ADMM-Net: A Deep Learning Approach for Compressive Sensing MRI , 2017, ArXiv.

[53]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[54]  G. Elaiyaraja,et al.  Fast and Efficient Filter Using Wavelet Threshold for Removal of Gaussian Noise from MRI/CT Scanned Medical Images/Color Video Sequence , 2019, IETE Journal of Research.

[55]  Yuxiang Xing,et al.  Comparision of projection domain, image domain, and comprehensive deep learning for sparse-view X-ray CT image reconstruction , 2018, 1804.04289.