FastPET: Near Real-Time Reconstruction of PET Histo-Image Data Using a Neural Network

Direct reconstruction of positron emission tomography (PET) data using deep neural networks is a growing field of research. Initial results are promising, but often the networks are complex, memory utilization inefficient, produce relatively small 2-D image slices (e.g., $128 \times 128$ ), and low count rate reconstructions are of varying quality. This article proposes FastPET, a novel direct reconstruction convolutional neural network that is architecturally simple, memory space efficient, works for nontrivial 3-D image volumes and is capable of processing a wide spectrum of PET data including low-dose and multitracer applications. FastPET uniquely operates on a histo-image (i.e., image-space) representation of the raw data enabling it to reconstruct 3-D image volumes $67\times $ faster than ordered subsets expectation maximization (OSEM). We detail the FastPET method trained on whole-body and low-dose whole-body data sets and explore qualitative and quantitative aspects of reconstructed images from clinical and phantom studies. Additionally, we explore the application of FastPET on a neurology data set containing multiple different tracers. The results show that not only are the reconstructions very fast, but the images are high quality and have lower noise than iterative reconstructions.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Tao Tan,et al.  Denoising of 3D magnetic resonance images with multi-channel residual learning of convolutional neural network , 2017, Japanese Journal of Radiology.

[3]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[4]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[5]  Stefaan Vandenberghe,et al.  Fast reconstruction of 3D time-of-flight PET data by axial rebinning and transverse mashing , 2006, Physics in medicine and biology.

[6]  Ronald Boellaard,et al.  Performance Characteristics of the Digital Biograph Vision PET/CT System , 2019, The Journal of Nuclear Medicine.

[7]  Jiasong Wu,et al.  Improving Low-Dose CT Image Using Residual Convolutional Network , 2017, IEEE Access.

[8]  Vladimir Y. Panin,et al.  Fully 3-D PET reconstruction with system matrix derived from point source measurements , 2006, IEEE Transactions on Medical Imaging.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jens Gregor,et al.  DirectPET: full-size neural network PET reconstruction from sinogram data , 2020, Journal of medical imaging.

[11]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[12]  Chih-Chieh Liu,et al.  PET Image Denoising Using a Deep Neural Network Through Fine Tuning , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Maria Argyrou,et al.  Tomographic Image Reconstruction based on Artificial Neural Network (ANN) techniques , 2012, 2012 IEEE Nuclear Science Symposium and Medical Imaging Conference Record (NSS/MIC).

[15]  Javad Alirezaie,et al.  Low-dose CT Denoising with Dilated Residual Network , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[17]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[18]  Thomas J. Fuchs,et al.  DeepPET: A deep encoder–decoder network for directly solving the PET image reconstruction inverse problem , 2018, Medical Image Anal..

[19]  Yang-Ming Zhu,et al.  Full-Dose PET Image Estimation from Low-Dose PET Image Using Deep Learning: a Pilot Study , 2018, Journal of Digital Imaging.

[20]  J. Bowsher,et al.  Artificial neural networks for SPECT image reconstruction with optimized weighted backprojection , 1991, Conference Record of the 1991 IEEE Nuclear Science Symposium and Medical Imaging Conference.

[21]  Yong Xu,et al.  Deep Learning for Image Denoising: A Survey , 2018, ICGEC.

[22]  M. Ter-pogossian,et al.  Image-Reconstruction of Data from Super PETT I: A First-Generation Time-of-Flight Positron-Emission Tomograph , 1986, IEEE Transactions on Nuclear Science.

[23]  Jong Chul Ye,et al.  Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT , 2017, IEEE Transactions on Medical Imaging.

[24]  Jaejun Yoo,et al.  Deep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network , 2018, IEEE Transactions on Medical Imaging.

[25]  Tianyu Ma,et al.  An investigation of quantitative accuracy for deep learning based denoising in oncological PET , 2019, Physics in medicine and biology.

[26]  Joel S. Karp,et al.  Efficient 3-D TOF PET Reconstruction Using View-Grouped Histo-Images: DIRECT—Direct Image Reconstruction for TOF , 2009, IEEE Transactions on Medical Imaging.

[27]  P. Miné,et al.  A new approach to positron emission tomography , 2004, European Journal of Nuclear Medicine.

[28]  H. Sebastian Seung,et al.  Natural Image Denoising with Convolutional Networks , 2008, NIPS.

[29]  Jeffrey A. Fessler,et al.  Image Reconstruction is a New Frontier of Machine Learning , 2018, IEEE Transactions on Medical Imaging.

[30]  Andrew J. Reader,et al.  Micro-Networks for Robust MR-Guided Low Count PET Imaging , 2020, IEEE Transactions on Radiation and Plasma Medical Sciences.

[31]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[32]  Luyao Shi,et al.  A Novel Loss Function Incorporating Imaging Acquisition Physics for PET Attenuation Map Generation using Deep Learning , 2019, MICCAI.

[33]  N. Giokaris,et al.  Tomographic image reconstruction using Artificial Neural Networks , 2004 .

[34]  Phaneendra K. Yalavarthy,et al.  Convolutional Neural Network-Based Robust Denoising of Low-Dose Computed Tomography Perfusion Maps , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Jieqing Jiao,et al.  Fast PET reconstruction using Multi-scale Fully Convolutional Neural Networks , 2017, ArXiv.

[37]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[38]  Guang Yang,et al.  Reduction of Gibbs artifacts in magnetic resonance imaging based on Convolutional Neural Network , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[39]  H. Malcolm Hudson,et al.  Accelerated image reconstruction using ordered subsets of projection data , 1994, IEEE Trans. Medical Imaging.