Current Clinical Applications of Artificial Intelligence in Radiology and Their Best Supporting Evidence.

PURPOSE Despite tremendous gains from deep learning and the promise of artificial intelligence (AI) in medicine to improve diagnosis and save costs, there exists a large translational gap to implement and use AI products in real-world clinical situations. Adoption of standards such as Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis, Consolidated Standards of Reporting Trials, and the Checklist for Artificial Intelligence in Medical Imaging is increasing to improve the peer-review process and reporting of AI tools. However, no such standards exist for product-level review. METHODS A review of clinical trials showed a paucity of evidence for radiology AI products; thus, the authors developed a 10-question assessment tool for reviewing AI products with an emphasis on their validation and result dissemination. The assessment tool was applied to commercial and open-source algorithms used for diagnosis to extract evidence on the clinical utility of the tools. RESULTS There is limited technical information on methodologies for FDA-approved algorithms compared with open-source products, likely because of intellectual property concerns. Furthermore, FDA-approved products use much smaller data sets compared with open-source AI tools, because the terms of use of public data sets are limited to academic and noncommercial entities, which precludes their use in commercial products. CONCLUSIONS Overall, this study reveals a broad spectrum of maturity and clinical use of AI products, but a large gap exists in exploring actual performance of AI tools in clinical practice.

[1]  S. Park,et al.  Design Characteristics of Studies Reporting the Performance of Artificial Intelligence Algorithms for Diagnostic Analysis of Medical Images: Results from Recently Published Papers , 2019, Korean journal of radiology.

[2]  Li Shen,et al.  Deep Learning to Improve Breast Cancer Detection on Screening Mammography , 2017, Scientific Reports.

[3]  Johannes B Reitsma,et al.  STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration , 2016, BMJ Open.

[4]  David S. Melnick,et al.  International evaluation of an AI system for breast cancer screening , 2020, Nature.

[5]  A. Localio,et al.  TRIPOD: A New Reporting Baseline for Developing and Interpreting Prediction Models , 2015, Annals of Internal Medicine.

[6]  Stephen T. C. Wong,et al.  A Deep Learning-Based Decision Support Tool for Precision Risk Assessment of Breast Cancer. , 2019, JCO clinical cancer informatics.

[7]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[8]  Daniel L Rubin,et al.  A curated mammography data set for use in computer-aided detection and diagnosis research , 2017, Scientific Data.

[9]  Marcus A. Badgeley,et al.  Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study , 2018, PLoS medicine.

[10]  Brian W. Powers,et al.  Dissecting racial bias in an algorithm used to manage the health of populations , 2019, Science.

[11]  J. Ioannidis,et al.  Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies , 2020, BMJ.

[12]  Nan Wu,et al.  Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening , 2019, IEEE Transactions on Medical Imaging.

[13]  Simukayi Mutasa,et al.  Global Trend in Artificial Intelligence-Based Publications in Radiology From 2000 to 2018. , 2019, AJR. American journal of roentgenology.

[14]  Matthew P. Lungren,et al.  PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging , 2020, npj Digital Medicine.

[15]  R. Barzilay,et al.  A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction. , 2019, Radiology.

[16]  Derek Merck,et al.  Generalizable Inter-Institutional Classification of Abnormal Chest Radiographs Using Efficient Convolutional Neural Networks , 2019, Journal of Digital Imaging.

[17]  Brandon K. Fornwalt,et al.  Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration , 2018, npj Digital Medicine.

[18]  Jitendra Malik,et al.  Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning , 2019, Proceedings of the National Academy of Sciences.