Artificial intelligence, machine (deep) learning and radio(geno)mics: definitions and nuclear medicine imaging applications

Techniques from the field of artificial intelligence, and more specifically machine (deep) learning methods, have been core components of most recent developments in the field of medical imaging. They are already being exploited or are being considered to tackle most tasks, including image reconstruction, processing (denoising, segmentation), analysis and predictive modelling. In this review we introduce and define these key concepts and discuss how the techniques from this field can be applied to nuclear medicine imaging applications with a particular focus on radio(geno)mics.

[1]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[2]  Ahmed Hosny,et al.  Artificial intelligence in radiology , 2018, Nature Reviews Cancer.

[3]  E. Pakdemirli.  Artificial intelligence in radiology: friend or foe? Where are we now and where are we heading? , 2019, Acta radiologica open.

[4]  R. Gillies,et al.  Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study , 2018, PLoS medicine.

[5]  Philipp A. Kaufmann,et al.  Automated detection of lung cancer at ultralow dose PET/CT by deep neural networks - Initial results. , 2018, Lung cancer.

[6]  Wei Lu,et al.  Tumor co-segmentation in PET/CT using multi-modality fully convolutional neural network , 2018, Physics in medicine and biology.

[7]  Dimitris Visvikis,et al.  Denoising of PET images by combining wavelets and curvelets for improved preservation of resolution and quantitation , 2013, Medical Image Anal..

[8]  Caroline Reinhold,et al.  Creating Robust Predictive Radiomic Models for Data From Independent Institutions Using Normalization , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[9]  Jong Hoon Kim,et al.  Penalized PET Reconstruction Using Deep Learning Prior and Local Linear Fitting , 2018, IEEE Transactions on Medical Imaging.

[10]  A. Amyar,et al.  3-D RPET-NET: Development of a 3-D PET Imaging Convolutional Neural Network for Radiomics Analysis and Outcome Prediction , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[11]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[12]  Timothy Solberg,et al.  Machine learning algorithms for outcome prediction in (chemo)radiotherapy: An empirical comparison of classifiers , 2018, Medical physics.

[13]  M. Hatt,et al.  Responsible Radiomics Research for Faster Clinical Translation , 2017, The Journal of Nuclear Medicine.

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Mark A. Anastasio,et al.  Treatment Outcome Prediction for Cancer Patients Based on Radiomics and Belief Function Theory , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[16]  Robert J. Gillies,et al.  Quantitative imaging of cancer in the postgenomic era: Radio(geno)mics, deep learning, and habitats , 2018, Cancer.

[17]  Tara N. Sainath,et al.  Improving deep neural networks for LVCSR using rectified linear units and dropout , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[19]  Yang-Ming Zhu,et al.  Full-Dose PET Image Estimation from Low-Dose PET Image Using Deep Learning: a Pilot Study , 2018, Journal of Digital Imaging.

[20]  F. Turkheimer,et al.  Evaluation of a 3D local multiresolution algorithm for the correction of partial volume effects in positron emission tomography. , 2011, Medical physics.

[21]  Howard Y. Chang,et al.  Decoding global gene expression programs in liver cancer by noninvasive imaging , 2007, Nature Biotechnology.

[22]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[23]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[24]  M. Hatt,et al.  IBSI: an international community radiomics standardization initiative , 2018 .

[25]  Rostom Mabrouk,et al.  Machine Learning Based Classification Using Clinical and DaTSCAN SPECT Imaging Features: A Study on Parkinson’s Disease and SWEDD , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[26]  Jae Sung Lee,et al.  Generation of PET Attenuation Map for Whole-Body Time-of-Flight 18F-FDG PET/MRI Using a Deep Neural Network Trained with Simultaneously Reconstructed Activity and Attenuation Maps , 2019, The Journal of Nuclear Medicine.

[27]  Xiang Li,et al.  Deep Learning-Based Image Segmentation on Multimodal Medical Imaging , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[28]  Jerzy W. Grzymala-Busse.  Artificial Intelligence - Introduction , 1993, ICCI.

[29]  S. Cherry,et al.  Innovations in Instrumentation for Positron Emission Tomography. , 2018, Seminars in nuclear medicine.

[30]  A. Rutman,et al.  Radiogenomics: creating a link between molecular diagnostics and diagnostic imaging. , 2009, European journal of radiology.

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  John O. Prior,et al.  Automatic lesion detection and segmentation of 18F-FET PET in gliomas: A full 3D U-Net convolutional neural network study , 2018, PloS one.

[33]  Caroline Reinhold,et al.  An Empirical Approach for Avoiding False Discoveries When Applying High-Dimensional Radiomics to Small Datasets , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[34]  Christos Davatzikos,et al.  Imaging genomics in cancer research: limitations and promises. , 2016, The British journal of radiology.

[35]  Wei Cao,et al.  Deep Learning based Radiomics (DLR) and its usage in noninvasive IDH1 prediction for low grade glioma , 2017, Scientific Reports.

[36]  M. Hatt,et al.  External validation of a combined PET and MRI radiomics model for prediction of recurrence in cervical cancer patients treated with chemoradiotherapy , 2018, European Journal of Nuclear Medicine and Molecular Imaging.

[37]  P. Lecoq,et al.  Pushing the Limits in Time-of-Flight PET Imaging , 2017, IEEE Transactions on Radiation and Plasma Medical Sciences.

[38]  Robert Jeraj,et al.  Automated classification of benign and malignant lesions in 18F-NaF PET/CT images using machine learning , 2018, Physics in medicine and biology.

[39]  M. Hatt,et al.  Radiomics in PET/CT: More Than Meets the Eye? , 2017, The Journal of Nuclear Medicine.

[40]  Leixin Zhou,et al.  Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks , 2019, Medical physics.

[41]  Patrick Granton,et al.  Radiomics: extracting more information from medical images using advanced feature analysis. , 2012, European journal of cancer.

[42]  David G. Stork,et al.  Pattern Classification , 1973 .

[43]  Anne Berger,et al.  Deep Learning and Radiomics predict complete response after neo-adjuvant chemoradiation for locally advanced rectal cancer , 2018, Scientific Reports.

[44]  James H. Moor,et al.  The Dartmouth College Artificial Intelligence Conference: The Next Fifty Years , 2006, AI Mag..

[45]  Roger Allan Ford,et al.  Privacy and Accountability in Black-Box Medicine , 2016 .

[46]  P. Lambin,et al.  Machine Learning methods for Quantitative Radiomic Biomarkers , 2015, Scientific Reports.

[47]  Yiming Ding,et al.  A Deep Learning Model to Predict a Diagnosis of Alzheimer Disease by Using 18F-FDG PET of the Brain. , 2019, Radiology.

[48]  A. Soricelli,et al.  Dixon-VIBE Deep Learning (DIVIDE) Pseudo-CT Synthesis for Pelvis PET/MR Attenuation Correction , 2018, The Journal of Nuclear Medicine.

[49]  Jae Sung Lee,et al.  Improving the Accuracy of Simultaneously Reconstructed Activity and Attenuation Maps Using Deep Learning , 2018, The Journal of Nuclear Medicine.

[50]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[51]  Robert J. Gillies,et al.  Predicting Nodule Malignancy using a CNN Ensemble Approach , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[52]  Dorit Merhof,et al.  Radiomic versus Convolutional Neural Networks Analysis for Classification of Contrast-enhancing Lesions at Multiparametric Breast MRI. , 2019, Radiology.

[53]  Chih-Chieh Liu,et al.  PET Image Denoising Using a Deep Neural Network Through Fine Tuning , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[54]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[55]  S. Cherry,et al.  Using convolutional neural networks to estimate time-of-flight from PET detector waveforms , 2018, Physics in medicine and biology.

[56]  A. McMillan,et al.  Deep learning Mr imaging–based attenuation correction for PeT/Mr imaging 1 , 2017 .

[57]  Yaozong Gao,et al.  Estimating CT Image from MRI Data Using 3D Fully Convolutional Networks , 2016, LABELS/DLMIA@MICCAI.

[58]  Andrew P. Leynes,et al.  Zero-Echo-Time and Dixon Deep Pseudo-CT (ZeDD CT): Direct Generation of Pseudo-CT Images for Pelvic PET/MRI Attenuation Correction Using Deep Convolutional Neural Networks with Multiparametric MRI , 2017, The Journal of Nuclear Medicine.

[59]  Andreas M. Kaplan,et al.  Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence , 2019, Business Horizons.

[60]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[61]  Zhen Lin,et al.  Choosing SNPs using feature selection , 2005, 2005 IEEE Computational Systems Bioinformatics Conference (CSB'05).

[62]  Hugo J. W. L. Aerts.  Radiomics: there is more than meets the eye in medical imaging (Conference Presentation) , 2016 .

[63]  V. Goh,et al.  Predicting Response to Neoadjuvant Chemotherapy with PET Imaging Using Convolutional Neural Networks , 2015, PloS one.

[64]  Christian Barillot,et al.  The first MICCAI challenge on PET tumor segmentation , 2018, Medical Image Anal..

[65]  Qiu Huang,et al.  Enhancing the Image Quality via Transferred Deep Residual Learning of Coarse PET Sinograms , 2018, IEEE Transactions on Medical Imaging.

[66]  Thomas Frauenfelder,et al.  Quantitative imaging. , 2015, Investigative radiology.

[67]  Lubomir M. Hadjiiski,et al.  Bladder Cancer Treatment Response Assessment in CT using Radiomics with Deep-Learning , 2017, Scientific Reports.

[68]  Arman Rahmim,et al.  Use of Generative Disease Models for Analysis and Selection of Radiomic Features in PET , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[69]  Paul Kinahan,et al.  A combined PET/CT scanner for clinical oncology. , 2000, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[70]  R. Gillies,et al.  The biology underlying molecular imaging in oncology: from genome to anatome and back again. , 2010, Clinical radiology.

[71]  Habib Zaidi,et al.  Classification and evaluation strategies of auto‐segmentation approaches for PET: Report of AAPM task group No. 211 , 2017, Medical physics.

[72]  Quanzheng Li,et al.  Iterative PET Image Reconstruction Using Convolutional Neural Network Representation , 2017, IEEE Transactions on Medical Imaging.

[73]  Caroline Reinhold,et al.  Comparison of Radiomics Models Built Through Machine Learning in a Multicentric Context With Independent Testing: Identical Data, Similar Algorithms, Different Methodologies , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[74]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[75]  Simukayi Mutasa,et al.  Convolutional Neural Network Using a Breast MRI Tumor Dataset Can Predict Oncotype Dx Recurrence Score , 2018, Journal of magnetic resonance imaging : JMRI.

[76]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[77]  Tanveer F. Syeda-Mahmood,et al.  Building medical image classifiers with very limited data using segmentation networks , 2018, Medical Image Anal..

[78]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[79]  Steffen Löck,et al.  A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling , 2017, Scientific Reports.

[80]  M. Hatt,et al.  Machine (Deep) Learning Methods for Image Processing and Radiomics , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.