CheXaid: deep learning assistance for physician diagnosis of tuberculosis using chest x-rays in patients with HIV

Tuberculosis (TB) is the leading cause of preventable death in HIV-positive patients, and yet often remains undiagnosed and untreated. Chest x-ray is often used to assist in diagnosis, yet this presents additional challenges due to atypical radiographic presentation and radiologist shortages in regions where co-infection is most common. We developed a deep learning algorithm to diagnose TB using clinical information and chest x-ray images from 677 HIV-positive patients with suspected TB from two hospitals in South Africa. We then sought to determine whether the algorithm could assist clinicians in the diagnosis of TB in HIV-positive patients as a web-based diagnostic assistant. Use of the algorithm resulted in a modest but statistically significant improvement in clinician accuracy ( p  = 0.002), increasing the mean clinician accuracy from 0.60 (95% CI 0.57, 0.63) without assistance to 0.65 (95% CI 0.60, 0.70) with assistance. However, the accuracy of assisted clinicians was significantly lower ( p  < 0.001) than that of the stand-alone algorithm, which had an accuracy of 0.79 (95% CI 0.77, 0.82) on the same unseen test cases. These results suggest that deep learning assistance may improve clinician accuracy in TB diagnosis using chest x-rays, which would be valuable in settings with a high burden of HIV/TB co-infection. Moreover, the high accuracy of the stand-alone algorithm suggests a potential value particularly in settings with a scarcity of radiological expertise.

[1]  R. Wood,et al.  Pulmonary tuberculosis in HIV infection: radiographic appearance is related to CD4+ T-lymphocyte count. , 1995, Tubercle and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease.

[2]  C. Whitty,et al.  'Smear-negative' pulmonary tuberculosis in a DOTS programme: poor outcomes in an area of high HIV seroprevalence. , 2001, The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease.

[3]  W. V. de Souza,et al.  Radiographic features of pulmonary tuberculosis in patients infected by HIV: is there an objective indicator of co-infection? , 2001, Revista da Sociedade Brasileira de Medicina Tropical.

[4]  Reuben Granich,et al.  HIV infection-associated tuberculosis: the epidemiology and the response. , 2010, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[5]  E. Bateman,et al.  Chest radiograph reading and recording system: evaluation for tuberculosis screening in patients with advanced HIV. , 2010, The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease.

[6]  A. Sharma,et al.  Manifestations of tuberculosis in HIV/AIDS patients and its relationship with CD4 count , 2011, Lung India : official organ of Indian Chest Society.

[7]  D. Alland,et al.  Xpert MTB/RIF: a New Pillar in Diagnosis of Extrapulmonary Tuberculosis? , 2011, Journal of Clinical Microbiology.

[8]  P. Glaziou,et al.  Assessing Tuberculosis Case Fatality Ratio: A Meta-Analysis , 2011, PloS one.

[9]  K. Dheda,et al.  Diagnostic accuracy of a urine lipoarabinomannan strip-test for TB detection in HIV-infected hospitalised patients , 2012, European Respiratory Journal.

[10]  A. Nawaz,et al.  HIV-Tuberculosis: A Study of Chest X-Ray Patterns in Relation to CD4 Count , 2012, North American journal of medical sciences.

[11]  A. Karargyris,et al.  Automatic screening for tuberculosis in chest radiographs: a survey. , 2013, Quantitative imaging in medicine and surgery.

[12]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[13]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[14]  B. van Ginneken,et al.  The Sensitivity and Specificity of Using a Computer Aided Diagnosis Program for Automatically Scoring Chest X-Rays of Presumptive TB Patients Compared with Xpert MTB/RIF in Lusaka Zambia , 2014, PloS one.

[15]  Clement J. McDonald,et al.  Automatic Tuberculosis Screening Using Chest Radiographs , 2014, IEEE Transactions on Medical Imaging.

[16]  N. Dendukuri,et al.  Xpert® MTB/RIF assay for pulmonary tuberculosis and rifampicin resistance in adults , 2014, The Cochrane database of systematic reviews.

[17]  B. van Ginneken,et al.  Automated chest-radiography as a triage for Xpert testing in resource-constrained settings: a prospective study of diagnostic accuracy and costs , 2015, Scientific Reports.

[18]  S. Lawn,et al.  Prevalence of tuberculosis in post-mortem studies of HIV-infected adults and children in resource-limited settings: a systematic review and meta-analysis , 2015, AIDS.

[19]  C. Heilig,et al.  Performance of Clinical Screening Algorithms for Tuberculosis Intensified Case Finding among People Living with HIV in Western Kenya , 2016, PloS one.

[20]  B. van Ginneken,et al.  An automated tuberculosis screening strategy combining X-ray-based computer-aided detection and clinical information , 2016, Scientific Reports.

[21]  P. Lakhani,et al.  Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks. , 2017, Radiology.

[22]  Z. Qin,et al.  An evaluation of automated chest radiography reading software for tuberculosis screening among public- and private-sector patients , 2017, European Respiratory Journal.

[23]  M. Yotebieng,et al.  Low implementation of Xpert MTB/RIF among HIV/TB co-infected adults in the International epidemiologic Databases to Evaluate AIDS (IeDEA) program , 2017, PloS one.

[24]  M. Pai,et al.  Market penetration of Xpert MTB/RIF in high tuberculosis burden countries: A trend analysis from 2014 - 2016. , 2018, Gates open research.

[25]  Eui Jin Hwang,et al.  Development and Validation of a Deep Learning–based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs , 2018, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[26]  Zhi Zhen Qin,et al.  Market penetration of Xpert MTB/RIF in high tuberculosis burden countries: A trend analysis from 2014 - 2016 , 2018, Gates open research.

[27]  A. Ng,et al.  Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet , 2018, PLoS medicine.

[28]  A. Kengne,et al.  Optimizing Tuberculosis Diagnosis in Human Immunodeficiency Virus–Infected Inpatients Meeting the Criteria of Seriously Ill in the World Health Organization Algorithm , 2018, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[29]  B. van Ginneken,et al.  Evaluation of the diagnostic accuracy of Computer-Aided Detection of tuberculosis on Chest radiography among private sector patients in Pakistan , 2018, Scientific Reports.

[30]  Simon Hunter,et al.  Randomized controlled trial of a 12-week digital care program in improving low back pain , 2019, npj Digital Medicine.

[31]  Z. Qin,et al.  Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems , 2019, Scientific Reports.

[32]  Aldenor G. Santos,et al.  Occurrence of the potent mutagens 2- nitrobenzanthrone and 3-nitrobenzanthrone in fine airborne particles , 2019, Scientific Reports.

[33]  Junzhi Wang,et al.  Responsive Cells for rhEGF bioassay Obtained through Screening of a CRISPR/Cas9 Library , 2019, Scientific Reports.

[34]  Rongguo Zhang,et al.  Evaluating a Fully Automated Pulmonary Nodule Detection Approach and Its Impact on Radiologist Performance. , 2019, Radiology. Artificial intelligence.

[35]  R. Barzilay,et al.  A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction. , 2019, Radiology.

[36]  A. J. Mariano,et al.  Human–machine partnership with artificial intelligence for chest radiograph diagnosis , 2019, npj Digital Medicine.

[37]  A. Ng,et al.  Deep Learning–Assisted Diagnosis of Cerebral Aneurysms Using the HeadXNet Model , 2019, JAMA network open.

[38]  Brief Report: Real-World Performance and Interobserver Agreement of Urine Lipoarabinomannan in Diagnosing HIV-Associated Tuberculosis in an Emergency Center , 2019, Journal of acquired immune deficiency syndromes.

[39]  Ryanne A. Brown,et al.  Impact of a deep learning assistant on the histopathologic classification of liver cancer , 2020, npj Digital Medicine.

[40]  Ryanne A. Brown,et al.  Impact of a deep learning assistant on the histopathologic classification of liver cancer. , 2020, NPJ digital medicine.

[41]  Andrew Y. Ng,et al.  CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting , 2020, ArXiv.

[42]  K. Dr,et al.  HIV-TUBERCULOSIS: A STUDY OF CHEST X-RAY PATTERNS IN RELATION TO CD4 COUNT , 2021, INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH.