On the role of artificial intelligence in medical imaging of COVID-19

Although a plethora of research articles on AI methods on COVID-19 medical imaging are published, their clinical value remains unclear. We conducted the largest systematic review of the literature addressing the utility of AI in imaging for COVID-19 patient care. By keyword searches on PubMed and preprint servers throughout 2020, we identified 463 manuscripts and performed a systematic meta-analysis to assess their technical merit and clinical relevance. Our analysis evidences a significant disparity between clinical and AI communities, both in the focus on imaging modalities (AI experts neglected CT and Ultrasound, favoring X-Ray) and performed tasks (71.9% of AI papers centered on diagnosis). The vast majority of manuscripts were found deficient regarding potential use in clinical practice, but 2.7% (N=12) publications were assigned a high maturity level and are summarized in greater detail. We provide an itemized discussion of the challenges in developing clinically relevant AI solutions with recommendations and remedies.

[1]  J. Timsit,et al.  Limited applicability of a COVID-19 specific mortality prediction rule to the intensive care setting , 2020, Nat. Mach. Intell..

[2]  Nikos Paragios,et al.  AI-driven quantification, staging and outcome prediction of COVID-19 pneumonia , 2020, Medical Image Analysis.

[3]  Lei Liu,et al.  Semi-Supervised Active Learning for COVID-19 Lung Ultrasound Multi-symptom Classification , 2020, 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI).

[4]  Noorbakhsh Amiri Golilarz,et al.  Blockchain-Federated-Learning and Deep Learning Models for COVID-19 Detection Using CT Imaging , 2021, IEEE Sensors Journal.

[5]  D. Dong,et al.  The Role of Imaging in the Detection and Management of COVID-19: A Review , 2020, IEEE Reviews in Biomedical Engineering.

[6]  C. Wagner,et al.  International collaboration during the COVID-19 crisis: autumn 2020 developments , 2020, Scientometrics.

[7]  H. Kauczor,et al.  The Role of Chest Imaging in Patient Management During the COVID-19 Pandemic , 2020, Chest.

[8]  Ruth M. J. Byrne,et al.  Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning , 2019, IJCAI.

[9]  Dinggang Shen,et al.  Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19 , 2020, IEEE Reviews in Biomedical Engineering.

[10]  M. J. Smith,et al.  Point‐of‐care lung ultrasound in patients with COVID‐19 – a narrative review , 2020, Anaesthesia.

[11]  Derek J. Wu,et al.  Development of a convolutional neural network to differentiate among the etiology of similar appearing pathological B lines on lung ultrasound: a deep learning study , 2021, BMJ Open.

[12]  Deepta Rajan,et al.  Improving Reliability of Clinical Models Using Prediction Calibration , 2020, UNSURE/GRAIL@MICCAI.

[13]  B. Stieltjes,et al.  Development and clinical implementation of tailored image analysis tools for COVID-19 in the midst of the pandemic: The synergetic effect of an open, clinically embedded software development platform and machine learning , 2020, European Journal of Radiology.

[14]  W. Liang,et al.  Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements, and Prognosis of COVID-19 Pneumonia Using Computed Tomography , 2020, Cell.

[15]  Heshui Shi,et al.  Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study , 2020, The Lancet Infectious Diseases.

[16]  L. Vetrugno,et al.  Our Italian experience using lung ultrasound for identification, grading and serial follow‐up of severity of lung involvement for management of patients with COVID‐19 , 2020, Echocardiography.

[17]  Lihua Li,et al.  A Rapid, Accurate and Machine-Agnostic Segmentation and Quantification Method for CT-Based COVID-19 Diagnosis , 2020, IEEE Transactions on Medical Imaging.

[18]  Correlation between Chest Computed Tomography and Lung Ultrasonography in Patients with Coronavirus Disease 2019 (COVID-19) , 2020, Ultrasound in Medicine & Biology.

[19]  Fred Prior,et al.  Role of Machine Learning Techniques to Tackle the COVID-19 Crisis: Systematic Review , 2020, JMIR medical informatics.

[20]  J. Cheung,et al.  POCUS in COVID-19: pearls and pitfalls , 2020, The Lancet Respiratory Medicine.

[21]  H. Krumholz,et al.  Extrapulmonary manifestations of COVID-19 , 2020, Nature Medicine.

[22]  Yang Liu,et al.  Federated Learning , 2019, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[23]  Q. Tao,et al.  Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases , 2020, Radiology.

[24]  Yifan Yu,et al.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison , 2019, AAAI.

[25]  H. Zaidi,et al.  Ultra-low-dose chest CT imaging of COVID-19 patients using a deep residual neural network , 2020, European Radiology.

[26]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[27]  G. Heinze,et al.  Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal , 2020, BMJ.

[28]  Dorin Comaniciu,et al.  Automated Quantification of CT Patterns Associated with COVID-19 from Chest CT , 2020, Radiology. Artificial intelligence.

[29]  Carlos A. Silva,et al.  On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. , 2020, Radiology. Artificial intelligence.

[30]  Caroline S. Wagner,et al.  International collaboration during the COVID-19 crisis: autumn 2020 developments , 2021, Scientometrics.

[31]  Karsten M. Borgwardt,et al.  Accelerating detection of lung pathologies with explainable ultrasound image analysis , 2021 .

[32]  W. Liang,et al.  Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements, and Prognosis of COVID-19 Pneumonia Using Computed Tomography , 2020, Cell.

[33]  J. Mendel,et al.  Current Concepts Imaging in COVID-19 and the Challenges for Low and Middle Income Countries , 2020, Journal of Global Radiology.

[34]  Roie Melamed,et al.  Predicting Breast Cancer by Applying Deep Learning to Linked Health Records and Mammograms. , 2019, Radiology.

[35]  Liyana Shuib,et al.  Early survey with bibliometric analysis on machine learning approaches in controlling COVID-19 outbreaks , 2020, PeerJ Comput. Sci..

[36]  H. Rajebi,et al.  CT Features and Short-term Prognosis of COVID-19 Pneumonia: A Single-Center Study from Kashan, Iran , 2020, Radiology. Cardiothoracic imaging.

[37]  D. Buonsenso,et al.  COVID-19 outbreak: less stethoscope, more ultrasound , 2020, The Lancet Respiratory Medicine.

[38]  J. Ioannidis,et al.  Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies , 2020, BMJ.

[39]  Feng Qian,et al.  The Value of Federated Learning During and Post COVID-19. , 2021, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[40]  Jianming Wang,et al.  AI-assisted CT imaging analysis for COVID-19 screening: Building and deploying a medical AI system , 2020, Applied Soft Computing.

[41]  Theodoros P. Zanos,et al.  External validation demonstrates limited clinical utility of the interpretable mortality prediction model for patients with COVID-19 , 2020, Nature Machine Intelligence.

[42]  Hui Shen,et al.  Dual-branch combination network (DCN): Towards accurate diagnosis and lesion segmentation of COVID-19 using CT images , 2020, Medical Image Analysis.

[43]  Mingzhi Li,et al.  Coronavirus Disease (COVID-19): Spectrum of CT Findings and Temporal Progression of the Disease , 2020, Academic Radiology.

[44]  A. Borakati,et al.  Diagnostic accuracy of X-ray versus CT in COVID-19: a propensity-matched database study , 2020, BMJ Open.

[45]  Jayashree Kalpathy-Cramer,et al.  Automated Assessment and Tracking of COVID-19 Pulmonary Disease Severity on Chest Radiographs using Convolutional Siamese Neural Networks , 2020, Radiology. Artificial intelligence.

[46]  Elie A Akl,et al.  Use of Chest Imaging in the Diagnosis and Management of COVID-19: A WHO Rapid Advice Guide , 2020, Radiology.

[47]  Z. Fayad,et al.  Artificial intelligence–enabled rapid diagnosis of patients with COVID-19 , 2020, Nature Medicine.

[48]  J. Pare,et al.  Point-of-care Lung Ultrasound Is More Sensitive than Chest Radiograph for Evaluation of COVID-19 , 2020, The western journal of emergency medicine.

[49]  Sachin Shetty,et al.  Integrating blockchain for data sharing and collaboration in mobile healthcare applications , 2017, 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).

[50]  Keith Sigel,et al.  An inflammatory cytokine signature predicts COVID-19 severity and survival , 2020, Nature Medicine.

[51]  M. Castillo,et al.  The Industry of CT Scanning , 2012, American Journal of Neuroradiology.

[52]  K. Cao,et al.  Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy , 2020 .

[53]  P. Lomoro,et al.  COVID-19 pneumonia manifestations at the admission on chest ultrasound, radiographs, and CT: single-center study and comprehensive radiologic literature review , 2020, European Journal of Radiology Open.

[54]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[55]  Wufeng Xue,et al.  Modality alignment contrastive learning for severity assessment of COVID-19 from lung ultrasound and clinical information , 2021, Medical Image Analysis.

[56]  Philipp Berens,et al.  Test-time Data Augmentation for Estimation of Heteroscedastic Aleatoric Uncertainty in Deep Neural Networks , 2018 .

[57]  Yifan Yang,et al.  Experiments of Federated Learning for COVID-19 Chest X-ray Images , 2020, Advances in Artificial Intelligence and Security.

[58]  D. Moher,et al.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement , 2009, BMJ : British Medical Journal.

[59]  Christian Etmann,et al.  Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans , 2020 .

[60]  Sebasti'an Amador S'anchez,et al.  Comparative study of deep learning methods for the automatic segmentation of lung, lesion and lesion type in CT scans of COVID-19 patients , 2020, ArXiv.

[61]  Leo Anthony Celi,et al.  An awakening in medicine: the partnership of humanity and intelligent machines. , 2019, The Lancet. Digital health.

[62]  M. Kuo,et al.  Frequency and Distribution of Chest Radiographic Findings in COVID-19 Positive Patients , 2019, Radiology.

[63]  Tanaka.  The role of , 2000, Journal of insect physiology.

[64]  S. Scarlata,et al.  COVID‐19 diagnosis and management: a comprehensive review , 2020, Journal of internal medicine.

[65]  Plamen Angelov,et al.  SARS-CoV-2 CT-scan dataset:A large dataset of real patients CT scans for SARS-CoV-2 identification , 2020 .

[66]  Carl E. Rasmussen,et al.  The Need for Open Source Software in Machine Learning , 2007, J. Mach. Learn. Res..

[67]  Joseph Paul Cohen,et al.  COVID-19 Image Data Collection: Prospective Predictions Are the Future , 2020, ArXiv.

[68]  Ziyue Xu,et al.  Determination of disease severity in COVID-19 patients using deep learning in chest X-ray images. , 2020, Diagnostic and interventional radiology.

[69]  Louise Wilkinson,et al.  OPTIMAM Mammography Image Database: a large scale resource of mammography images and clinical data , 2020, Radiology. Artificial intelligence.

[70]  F J Gilbert,et al.  Artificial intelligence in breast imaging. , 2019, Clinical radiology.

[71]  Yvonne Vergouwe,et al.  Towards better clinical prediction models: seven steps for development and an ABCD for validation. , 2014, European heart journal.

[72]  M. Quanjel,et al.  Replication of a mortality prediction model in Dutch patients with COVID-19 , 2020, Nat. Mach. Intell..

[73]  K. Yeom,et al.  End-to-end automatic differentiation of the coronavirus disease 2019 (COVID-19) from viral pneumonia based on chest CT , 2020, European Journal of Nuclear Medicine and Molecular Imaging.

[74]  S. Reeder,et al.  Diagnosis of COVID-19 Pneumonia Using Chest Radiography: Value of Artificial Intelligence , 2020 .

[75]  Jasjit S. Suri,et al.  The present and future of deep learning in radiology. , 2019, European journal of radiology.

[76]  Mustafa Suleyman,et al.  Key challenges for delivering clinical impact with artificial intelligence , 2019, BMC Medicine.

[77]  David S. Melnick,et al.  International evaluation of an AI system for breast cancer screening , 2020, Nature.

[78]  Jonathan E. Schmitz,et al.  Laboratory Diagnosis of COVID-19: Current Issues and Challenges , 2020, Journal of Clinical Microbiology.

[79]  Y. Tung-Chen,et al.  Correlation between Chest Computed Tomography and Lung Ultrasonography in Patients with Coronavirus Disease 2019 (COVID-19) , 2020, Ultrasound in Medicine & Biology.

[80]  Jianjiang Feng,et al.  Development and evaluation of an artificial intelligence system for COVID-19 diagnosis , 2020, Nature Communications.

[81]  T. Olgers,et al.  Diagnosing COVID-19 pneumonia in a pandemic setting: Lung Ultrasound versus CT (LUVCT) – a multicentre, prospective, observational study , 2020, ERJ Open Research.

[82]  H. Zhao,et al.  6 Lung Ultrasound versus Chest X-ray for the Diagnosis of COVID-19 Pneumonia , 2020, Annals of Emergency Medicine.

[83]  L. Mombaerts,et al.  An interpretable mortality prediction model for COVID-19 patients , 2020, Nature Machine Intelligence.

[84]  Hyo-Eun Kim,et al.  Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. , 2020, The Lancet. Digital health.

[85]  Nikolas Lessmann,et al.  Automated Assessment of CO-RADS and Chest CT Severity Scores in Patients with Suspected COVID-19 Using Artificial Intelligence , 2020, Radiology.

[86]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[87]  M. Jorge Cardoso,et al.  Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis: Second International Workshop, UNSURE 2020, and Third International Workshop, GRAIL 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 8, 2020, Proceedings , 2020, UNSURE/GRAIL@MICCAI.

[88]  Yun Yang,et al.  Comparison and Modelling of Country-level Microblog User and Activity in Cyber-physical-social Systems Using Weibo and Twitter Data , 2019, ACM Trans. Intell. Syst. Technol..

[89]  Doina Precup,et al.  Exploring uncertainty measures in deep networks for Multiple sclerosis lesion detection and segmentation , 2020, Medical Image Anal..

[90]  Zhiyong Xu,et al.  A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images , 2020, IEEE Transactions on Medical Imaging.

[91]  Berkman Sahiner,et al.  Evaluation of Combined Artificial Intelligence and Radiologist Assessment to Interpret Screening Mammograms , 2020, JAMA network open.

[92]  G. Corrado,et al.  End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography , 2019, Nature Medicine.

[93]  Trevor Hastie,et al.  Transparency and reproducibility in artificial intelligence , 2020, Nature.

[94]  B. van Ginneken,et al.  Relational Modeling for Robust and Efficient Pulmonary Lobe Segmentation in CT Scans. , 2020, IEEE transactions on medical imaging.

[95]  Raymond Y Huang,et al.  Artificial Intelligence Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Origin at Chest CT , 2021, Radiology.

[96]  Karrar Hameed Abdulkareem,et al.  Systematic review of artificial intelligence techniques in the detection and classification of COVID-19 medical images in terms of evaluation and benchmarking: Taxonomy analysis, challenges, future solutions and methodological aspects , 2020, Journal of Infection and Public Health.

[97]  Milos Hauskrecht,et al.  Learning classification models with soft-label information , 2014, J. Am. Medical Informatics Assoc..

[98]  Kipp W. Johnson,et al.  Federated Learning of Electronic Health Records to Improve Mortality Prediction in Hospitalized Patients With COVID-19: Machine Learning Approach , 2020, JMIR medical informatics.

[99]  Karsten M. Borgwardt,et al.  Accelerating COVID-19 Differential Diagnosis with Explainable Ultrasound Image Analysis , 2020, ArXiv.