Next generation research applications for hybrid PET/MR and PET/CT imaging using deep learning

IntroductionRecently there have been significant advances in the field of machine learning and artificial intelligence (AI) centered around imaging-based applications such as computer vision. In particular, the tremendous power of deep learning algorithms, primarily based on convolutional neural network strategies, is becoming increasingly apparent and has already had direct impact on the fields of radiology and nuclear medicine. While most early applications of computer vision to radiological imaging have focused on classification of images into disease categories, it is also possible to use these methods to improve image quality. Hybrid imaging approaches, such as PET/MRI and PET/CT, are ideal for applying these methods.MethodsThis review will give an overview of the application of AI to improve image quality for PET imaging directly and how the additional use of anatomic information from CT and MRI can lead to further benefits. For PET, these performance gains can be used to shorten imaging scan times, with improvement in patient comfort and motion artifacts, or to push towards lower radiotracer doses. It also opens the possibilities for dual tracer studies, more frequent follow-up examinations, and new imaging indications. How to assess quality and the potential effects of bias in training and testing sets will be discussed.ConclusionHarnessing the power of these new technologies to extract maximal information from hybrid PET imaging will open up new vistas for both research and clinical applications with associated benefits in patient care.

[1]  Richard Kijowski,et al.  A deep learning approach for 18F-FDG PET attenuation correction , 2018, EJNMMI Physics.

[2]  C. Langlotz,et al.  Deep Learning in Neuroradiology , 2018, American Journal of Neuroradiology.

[3]  Andrew P. Leynes,et al.  Zero-Echo-Time and Dixon Deep Pseudo-CT (ZeDD CT): Direct Generation of Pseudo-CT Images for Pelvic PET/MRI Attenuation Correction Using Deep Convolutional Neural Networks with Multiparametric MRI , 2017, The Journal of Nuclear Medicine.

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Hesamoddin Jahanian,et al.  Comparison of cerebral blood flow measurement with [15O]-water positron emission tomography and arterial spin labeling magnetic resonance imaging: A systematic review , 2016, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[6]  Geoffrey E. Hinton.  Deep Learning-A Technology With the Potential to Transform Health Care. , 2018, JAMA.

[7]  Quanzheng Li,et al.  Attenuation correction for brain PET imaging using deep neural network based on Dixon and ZTE MR images , 2017, Physics in medicine and biology.

[8]  S. Holm,et al.  Region specific optimization of continuous linear attenuation coefficients based on UTE (RESOLUTE): application to PET/MR brain imaging , 2015, Physics in medicine and biology.

[9]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[10]  A. McMillan,et al.  Feasibility of Deep Learning–Based PET/MR Attenuation Correction in the Pelvis Using Only Diagnostic MR Images , 2018, Tomography.

[11]  A. Ng,et al.  Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists , 2018, PLoS medicine.

[12]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[13]  Greg Zaharchuk,et al.  Ultra-low-dose PET reconstruction using generative adversarial network with feature matching and task-specific perceptual loss. , 2019, Medical physics.

[14]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[15]  Josef A. Lundman,et al.  Zero TE‐based pseudo‐CT image conversion in the head and its application in PET/MR attenuation correction and MR‐guided radiation therapy planning , 2018, Magnetic resonance in medicine.

[16]  Gudrun Wagenknecht,et al.  MRI for attenuation correction in PET: methods and challenges , 2012, Magnetic Resonance Materials in Physics, Biology and Medicine.

[17]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[18]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[19]  Junshen Xu,et al.  200x Low-dose PET Reconstruction using Deep Learning , 2017, ArXiv.

[20]  John M Pauly,et al.  Ultra-Low-Dose 18F-Florbetaben Amyloid PET Imaging Using Deep Learning with Multi-Contrast MRI Inputs. , 2019, Radiology.

[21]  Faraz Oloumi,et al.  Deep learning in ophthalmology: a review. , 2018, Canadian journal of ophthalmology. Journal canadien d'ophtalmologie.

[22]  Thomas J. Fuchs,et al.  DeepPET: A deep encoder–decoder network for directly solving the PET image reconstruction inverse problem , 2018, Medical Image Anal..

[23]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[24]  Dinggang Shen,et al.  Deep auto-context convolutional neural networks for standard-dose PET image estimation from low-dose PET/MRI , 2017, Neurocomputing.

[25]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[26]  Yi Su,et al.  Impact of MR-Based Attenuation Correction on Neurologic PET Studies , 2016, The Journal of Nuclear Medicine.

[27]  Ninon Burgos,et al.  A multi-centre evaluation of eleven clinically feasible brain PET/MRI attenuation correction techniques using a large cohort of patients , 2016, NeuroImage.

[28]  Andrew P. Leynes,et al.  Hybrid ZTE/Dixon MR‐based attenuation correction for quantitative uptake estimation of pelvic lesions in PET/MRI , 2017, Medical physics.

[29]  A. McMillan,et al.  Deep learning Mr imaging–based attenuation correction for PeT/Mr imaging 1 , 2017 .

[30]  Mannudeep K. Kalra,et al.  Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN) , 2017, ArXiv.

[31]  Nicholas Ayache,et al.  Learning Myelin Content in Multiple Sclerosis from Multimodal MRI through Adversarial Training , 2018, MICCAI.

[32]  Morteza Mardani,et al.  Deep Generative Adversarial Neural Networks for Compressive Sensing MRI , 2019, IEEE Transactions on Medical Imaging.

[33]  C. Langlotz,et al.  Performance of a Deep-Learning Neural Network Model in Assessing Skeletal Maturity on Pediatric Hand Radiographs. , 2017, Radiology.