Development and validation of the 4C Deterioration model for adults hospitalised with COVID-19

Prognostic models to predict the risk of clinical deterioration in acute COVID-19 are required to inform clinical management decisions. Among 75,016 consecutive adults across England, Scotland and Wales prospectively recruited to the ISARIC Coronavirus Clinical Characterisation Consortium (ISARIC4C) study, we developed and validated a multivariable logistic regression model for in-hospital clinical deterioration (defined as any requirement of ventilatory support or critical care, or death) using 11 routinely measured variables. We used internal-external cross-validation to show consistent measures of discrimination, calibration and clinical utility across eight geographical regions. We further validated the final model in held-out data from 8,252 individuals in London, with similarly consistent performance (C-statistic 0.77 (95% CI 0.75 to 0.78); calibration-in-the-large 0.01 (-0.04 to 0.06); calibration slope 0.96 (0.90 to 1.02)). Importantly, this model demonstrated higher net benefit than using other candidate scores to inform decision-making. Our 4C Deterioration model thus demonstrates unprecedented clinical utility and generalisability to predict clinical deterioration among adults hospitalised with COVID-19.

[1]  B. Furlow COVACTA trial raises questions about tocilizumab's benefit in COVID-19 , 2020, The Lancet Rheumatology.

[2]  Arthur S Slutsky,et al.  Association Between Administration of Systemic Corticosteroids and Mortality Among Critically Ill Patients With COVID-19: A Meta-analysis. , 2020, JAMA.

[3]  Keith Sigel,et al.  An inflammatory cytokine signature predicts COVID-19 severity and survival , 2020, Nature Medicine.

[4]  Jordan J. Clark,et al.  Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score , 2020, BMJ.

[5]  Mahdad Noursadeghi,et al.  Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study , 2020, European Respiratory Journal.

[6]  Lee-Jen Wei,et al.  Remdesivir for the Treatment of Covid-19 - Preliminary Report. , 2020, The New England journal of medicine.

[7]  Jennifer L. Bell,et al.  Effect of Dexamethasone in Hospitalized Patients with COVID-19: Preliminary Report , 2020, medRxiv.

[8]  Mike Clarke,et al.  A minimal common outcome measure set for COVID-19 clinical research , 2020, The Lancet Infectious Diseases.

[9]  P. Horby,et al.  Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study , 2020, BMJ.

[10]  C. Li,et al.  Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK , 2020, medRxiv.

[11]  G. Heinze,et al.  Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal , 2020, BMJ.

[12]  Richard D Riley,et al.  Temporal recalibration for improving prognostic model development and risk predictions in settings where survival is improving over time , 2020, International journal of epidemiology.

[13]  Richard D Riley,et al.  Calculating the sample size required for developing a clinical prediction model , 2020, BMJ.

[14]  Jian Sun,et al.  ACP risk grade: a simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China , 2020, medRxiv.

[15]  Kohske Takahashi,et al.  Welcome to the Tidyverse , 2019, J. Open Source Softw..

[16]  Andrew J. Vickers,et al.  A simple, step-by-step guide to interpreting decision curve analysis , 2019, Diagnostic and Prognostic Research.

[17]  Richard D Riley,et al.  Minimum sample size for developing a multivariable prediction model: PART II ‐ binary and time‐to‐event outcomes , 2018, Statistics in medicine.

[18]  G. Collins,et al.  PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies , 2019, Annals of Internal Medicine.

[19]  J. Richard,et al.  Assessment of clinical criteria for sepsis-was the cart put before the horse? , 2016, Journal of thoracic disease.

[20]  Lena Osterhagen,et al.  Multiple Imputation For Nonresponse In Surveys , 2016 .

[21]  Frank E. Harrell,et al.  Prediction models need appropriate internal, internal-external, and external validation. , 2016, Journal of clinical epidemiology.

[22]  Johannes B. Reitsma,et al.  Individual Participant Data (IPD) Meta-analyses of Diagnostic and Prognostic Modeling Studies: Guidance on Their Use , 2015, PLoS medicine.

[23]  G. Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement , 2015, BMJ : British Medical Journal.

[24]  Yvonne Vergouwe,et al.  Towards better clinical prediction models: seven steps for development and an ABCD for validation. , 2014, European heart journal.

[25]  J. Hedlund,et al.  Improvement of CRB-65 as a prognostic tool in adult patients with community-acquired pneumonia , 2014, BMJ Open Respiratory Research.

[26]  R. Evans European Centre for Disease Prevention and Control. , 2014, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[27]  Gary B. Smith,et al.  The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. , 2013, Resuscitation.

[28]  Stef van Buuren,et al.  MICE: Multivariate Imputation by Chained Equations in R , 2011 .

[29]  Patrick Royston,et al.  Multiple imputation using chained equations: Issues and guidance for practice , 2011, Statistics in medicine.

[30]  Douglas G Altman,et al.  Dichotomizing continuous predictors in multiple regression: a bad idea , 2006, Statistics in medicine.

[31]  N. Miyashita,et al.  The JRS guidelines for the management of community-acquired pneumonia in adults: an update and new recommendations. , 2006, Internal medicine.

[32]  L. Lind,et al.  Rapid Emergency Medicine score: a new prognostic tool for in‐hospital mortality in nonsurgical emergency department patients , 2004, Journal of internal medicine.

[33]  W. Lim,et al.  Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study , 2003, Thorax.

[34]  C. Subbe,et al.  Validation of a modified Early Warning Score in medical admissions. , 2001, QJM : monthly journal of the Association of Physicians.

[35]  C. Mackenzie,et al.  A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. , 1987, Journal of chronic diseases.