Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score

Abstract Objective To develop and validate a pragmatic risk score to predict mortality in patients admitted to hospital with coronavirus disease 2019 (covid-19). Design Prospective observational cohort study. Setting International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) World Health Organization (WHO) Clinical Characterisation Protocol UK (CCP-UK) study (performed by the ISARIC Coronavirus Clinical Characterisation Consortium—ISARIC-4C) in 260 hospitals across England, Scotland, and Wales. Model training was performed on a cohort of patients recruited between 6 February and 20 May 2020, with validation conducted on a second cohort of patients recruited after model development between 21 May and 29 June 2020. Participants Adults (age ≥18 years) admitted to hospital with covid-19 at least four weeks before final data extraction. Main outcome measure In-hospital mortality. Results 35 463 patients were included in the derivation dataset (mortality rate 32.2%) and 22 361 in the validation dataset (mortality rate 30.1%). The final 4C Mortality Score included eight variables readily available at initial hospital assessment: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, level of consciousness, urea level, and C reactive protein (score range 0-21 points). The 4C Score showed high discrimination for mortality (derivation cohort: area under the receiver operating characteristic curve 0.79, 95% confidence interval 0.78 to 0.79; validation cohort: 0.77, 0.76 to 0.77) with excellent calibration (validation: calibration-in-the-large=0, slope=1.0). Patients with a score of at least 15 (n=4158, 19%) had a 62% mortality (positive predictive value 62%) compared with 1% mortality for those with a score of 3 or less (n=1650, 7%; negative predictive value 99%). Discriminatory performance was higher than 15 pre-existing risk stratification scores (area under the receiver operating characteristic curve range 0.61-0.76), with scores developed in other covid-19 cohorts often performing poorly (range 0.63-0.73). Conclusions An easy-to-use risk stratification score has been developed and validated based on commonly available parameters at hospital presentation. The 4C Mortality Score outperformed existing scores, showed utility to directly inform clinical decision making, and can be used to stratify patients admitted to hospital with covid-19 into different management groups. The score should be further validated to determine its applicability in other populations. Study registration ISRCTN66726260

[1]  U. Hadi,et al.  National early warning score (NEWS) 2 predicts hospital mortality from COVID-19 patients , 2022, Annals of Medicine and Surgery.

[2]  Yuedong Yang,et al.  Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images , 2020, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[3]  Mahdad Noursadeghi,et al.  Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study , 2020, European Respiratory Journal.

[4]  C. A. Shaw,et al.  Ethnicity and Outcomes from COVID-19: The ISARIC CCP-UK Prospective Observational Cohort Study of Hospitalised Patients , 2020 .

[5]  Ross D. Williams,et al.  Seek COVER: Development and validation of a personalized risk calculator for COVID-19 outcomes in an international network , 2020, medRxiv.

[6]  P. Horby,et al.  Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study , 2020, BMJ.

[7]  Limin Ou,et al.  Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. , 2020, JAMA internal medicine.

[8]  C. Li,et al.  Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK , 2020, medRxiv.

[9]  Nan-shan Zhong,et al.  Cardiovascular comorbidity and its impact on patients with COVID-19 , 2020, European Respiratory Journal.

[10]  Iain B McInnes,et al.  Obesity a Risk Factor for Severe COVID-19 Infection: Multiple Potential Mechanisms. , 2020, Circulation.

[11]  D. Mathieu,et al.  High Prevalence of Obesity in Severe Acute Respiratory Syndrome Coronavirus‐2 (SARS‐CoV‐2) Requiring Invasive Mechanical Ventilation , 2020, Obesity.

[12]  G. Heinze,et al.  Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal , 2020, BMJ.

[13]  Richard D Riley,et al.  Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal , 2020, BMJ.

[14]  Jian-feng Xie,et al.  Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19 , 2020, medRxiv.

[15]  T. Guo,et al.  Prognostic value of C-reactive protein in patients with COVID-19 , 2020, medRxiv.

[16]  W. Haas,et al.  Influenza-associated pneumonia as reference to assess seriousness of coronavirus disease (COVID-19) , 2020, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[17]  Richard D Riley,et al.  Calculating the sample size required for developing a clinical prediction model , 2020, BMJ.

[18]  Wei Wang,et al.  Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis , 2020, European Respiratory Journal.

[19]  Nuno Ferreira,et al.  Estimation of risk factors for COVID-19 mortality - preliminary results , 2020, medRxiv.

[20]  Ting Yu,et al.  Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study , 2020, The Lancet Respiratory Medicine.

[21]  Richard D Riley,et al.  Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal , 2020 .

[22]  I. Buchan,et al.  North-South disparities in English mortality1965–2015: longitudinal population study , 2017, Journal of Epidemiology & Community Health.

[23]  I. Buchan,et al.  North-South disparities in English mortality 1965-2015: longitudinal population study. , 2017, Journal of epidemiology and community health.

[24]  A. Farcomeni,et al.  Expanded CURB-65: a new score system predicts severity of community-acquired pneumonia with superior efficiency , 2016, Scientific Reports.

[25]  R. Bellomo,et al.  The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). , 2016, JAMA.

[26]  Lena Osterhagen,et al.  Multiple Imputation For Nonresponse In Surveys , 2016 .

[27]  Gareth Ambler,et al.  How to develop a more accurate risk prediction model when there are few events , 2015, BMJ : British Medical Journal.

[28]  G. Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement , 2015, BMC Medicine.

[29]  Gary S Collins,et al.  Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration , 2015, Annals of Internal Medicine.

[30]  G. Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement , 2015, British Journal of Cancer.

[31]  J. Hedlund,et al.  Improvement of CRB-65 as a prognostic tool in adult patients with community-acquired pneumonia , 2014, BMJ Open Respiratory Research.

[32]  P. Horby,et al.  Open source clinical science for emerging infections , 2013, The Lancet Infectious Diseases.

[33]  Inmaculada Arostegui,et al.  Use of generalised additive models to categorise continuous variables in clinical prediction , 2013, BMC Medical Research Methodology.

[34]  I. Buchan,et al.  Trends in mortality from 1965 to 2008 across the English north-south divide: comparative observational study , 2011, BMJ : British Medical Journal.

[35]  R. Chan,et al.  Comparison of clinical characteristics and performance of pneumonia severity score and CURB-65 among younger adults, elderly and very old subjects , 2010, Thorax.

[36]  J. Mandrekar Receiver operating characteristic curve in diagnostic test assessment. , 2010, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[37]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[38]  M. Kenward,et al.  Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls , 2009, BMJ : British Medical Journal.

[39]  Amaia Bilbao,et al.  Prospective comparison of severity scores for predicting clinically relevant outcomes for patients hospitalized with community-acquired pneumonia. , 2009, Chest.

[40]  Rory Wolfe,et al.  SMART-COP: a tool for predicting the need for intensive respiratory or vasopressor support in community-acquired pneumonia. , 2008, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[41]  E. Elkin,et al.  Decision Curve Analysis: A Novel Method for Evaluating Prediction Models , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.

[42]  T. Welte,et al.  CRB‐65 predicts death from community‐acquired pneumonia * , 2006, Journal of internal medicine.

[43]  N. Miyashita,et al.  The JRS guidelines for the management of community-acquired pneumonia in adults: an update and new recommendations. , 2006, Internal medicine.

[44]  J. Vincent,et al.  The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure , 1996, Intensive Care Medicine.

[45]  W. Lim,et al.  Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study , 2003, Thorax.

[46]  M. Fine,et al.  A prediction rule to identify low-risk patients with community-acquired pneumonia. , 1997, The New England journal of medicine.

[47]  J. Concato,et al.  Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. , 1995, Journal of clinical epidemiology.

[48]  Peter Lynn,et al.  Multiple Imputation for Nonresponse in Surveys. , 1988 .

[49]  C. Mackenzie,et al.  A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. , 1987, Journal of chronic diseases.

[50]  E. Draper,et al.  APACHE II: A severity of disease classification system , 1985, Critical care medicine.