Autonomous Chest Radiograph Reporting Using AI: Estimation of Clinical Impact.

Background Automated interpretation of normal chest radiographs could alleviate the workload of radiologists. However, the performance of such an artificial intelligence (AI) tool compared with clinical radiology reports has not been established. Purpose To perform an external evaluation of a commercially available AI tool for (a) the number of chest radiographs autonomously reported, (b) the sensitivity for AI detection of abnormal chest radiographs, and (c) the performance of AI compared with that of the clinical radiology reports. Materials and Methods In this retrospective study, consecutive posteroanterior chest radiographs from adult patients in four hospitals in the capital region of Denmark were obtained in January 2020, including images from emergency department patients, in-hospital patients, and outpatients. Three thoracic radiologists labeled chest radiographs in a reference standard based on chest radiograph findings into the following categories: critical, other remarkable, unremarkable, or normal (no abnormalities). AI classified chest radiographs as high confidence normal (normal) or not high confidence normal (abnormal). Results A total of 1529 patients were included for analysis (median age, 69 years [IQR, 55-69 years]; 776 women), with 1100 (72%) classified by the reference standard as having abnormal radiographs, 617 (40%) as having critical abnormal radiographs, and 429 (28%) as having normal radiographs. For comparison, clinical radiology reports were classified based on the text and insufficient reports excluded (n = 22). The sensitivity of AI was 99.1% (95% CI: 98.3, 99.6; 1090 of 1100 patients) for abnormal radiographs and 99.8% (95% CI: 99.1, 99.9; 616 of 617 patients) for critical radiographs. Corresponding sensitivities for radiologist reports were 72.3% (95% CI: 69.5, 74.9; 779 of 1078 patients) and 93.5% (95% CI: 91.2, 95.3; 558 of 597 patients), respectively. Specificity of AI, and hence the potential autonomous reporting rate, was 28.0% of all normal posteroanterior chest radiographs (95% CI: 23.8, 32.5; 120 of 429 patients), or 7.8% (120 of 1529 patients) of all posteroanterior chest radiographs. Conclusion Of all normal posteroanterior chest radiographs, 28% were autonomously reported by AI with a sensitivity for any abnormalities higher than 99%. This corresponded to 7.8% of the entire posteroanterior chest radiograph production. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Park in this issue.

[1]  Alice C. Yu,et al.  External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review. , 2022, Radiology. Artificial intelligence.

[2]  J. Nam,et al.  Artificial intelligence system for identification of false-negative interpretations in chest radiographs , 2022, European Radiology.

[3]  B. van Ginneken,et al.  Artificial intelligence in radiology: 100 commercially available products and their scientific evidence , 2021, European Radiology.

[4]  R. Tappouni,et al.  Diagnosis of normal chest radiographs using an autonomous deep-learning algorithm. , 2021, Clinical radiology.

[5]  Po-Hsuan Cameron Chen,et al.  Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19 , 2020, Scientific Reports.

[6]  A. Ng,et al.  CheXaid: deep learning assistance for physician diagnosis of tuberculosis using chest x-rays in patients with HIV , 2020, npj Digital Medicine.

[7]  Sung Soo Lee,et al.  Deep Learning-based Automatic Detection Algorithm for Reducing Overlooked Lung Cancers on Chest Radiographs. , 2020, Radiology.

[8]  Jae Hyun Kim,et al.  Clinical Validation of a Deep Learning Algorithm for Detection of Pneumonia on Chest Radiographs in Emergency Department Patients with Acute Febrile Respiratory Illness , 2020, Journal of clinical medicine.

[9]  Zhiyong Lu,et al.  Automated abnormality classification of chest radiographs using deep convolutional neural networks , 2020, npj Digital Medicine.

[10]  J. Mongan,et al.  Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. , 2020, Radiology. Artificial intelligence.

[11]  Jin Mo Goo,et al.  Deep Learning for Chest Radiograph Diagnosis in the Emergency Department. , 2019, Radiology.

[12]  Eui Jin Hwang,et al.  Development and Validation of a Deep Learning–Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs , 2019, JAMA network open.

[13]  Jared A. Dunnmon,et al.  Assessment of Convolutional Neural Networks for Automated Classification of Chest Radiographs. , 2019, Radiology.

[14]  Mauro Annarumma,et al.  Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks. , 2019, Radiology.

[15]  Eui Jin Hwang,et al.  Development and Validation of Deep Learning-based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs. , 2019, Radiology.

[16]  David Moher,et al.  STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies , 2015, BMJ : British Medical Journal.

[17]  S. Raoof,et al.  Interpretation of plain chest roentgenogram. , 2012, Chest.

[18]  S. Carley,et al.  An introduction to power and sample size estimation , 2003, Emergency medicine journal : EMJ.