JURaSSiC: Accuracy of clinician vs risk score prediction of ischemic stroke outcomes

Objective: We compared the accuracy of clinicians and a risk score (iScore) to predict observed outcomes following an acute ischemic stroke. Methods: The JURaSSiC (Clinician JUdgment vs Risk Score to predict Stroke outComes) study assigned 111 clinicians with expertise in acute stroke care to predict the probability of outcomes of 5 ischemic stroke case scenarios. Cases (n = 1,415) were selected as being representative of the 10 most common clinical presentations from a pool of more than 12,000 stroke patients admitted to 12 stroke centers. The primary outcome was prediction of death or disability (modified Rankin Scale [mRS] ≥3) at discharge within the 95% confidence interval (CI) of observed outcomes. Secondary outcomes included 30-day mortality and death or institutionalization at discharge. Results: Clinicians made 1,661 predictions with overall accuracy of 16.9% for death or disability at discharge, 46.9% for 30-day mortality, and 33.1% for death or institutionalization at discharge. In contrast, 90% of the iScore-based estimates were within the 95% CI of observed outcomes. Nearly half (n = 53 of 111; 48%) of participants were unable to accurately predict the probability of the primary outcome in any of the 5 rated cases. Less than 1% (n = 1) provided accurate predictions in 4 of the 5 cases and none accurately predicted all 5 case outcomes. In multivariable analyses, the presence of patient characteristics associated with poor outcomes (mRS ≥3 or death) in previous studies (older age, high NIH Stroke Scale score, and nonlacunar subtype) were associated with more accurate clinician predictions of death at 30 days (odds ratio [OR] 2.40, 95% CI 1.57–3.67) and with a trend for more accurate predictions of death or disability at discharge (OR 1.85, 95% CI 0.99–3.46). Conclusions: Clinicians with expertise in stroke performed poorly compared to a validated tool in predicting the outcomes of patients with an acute ischemic stroke. Use of the risk stroke outcome tool may be superior for decision-making following an acute ischemic stroke.

[1]  Jill Beck,et al.  Training the clinical eye and mind: using the arts to develop medical students' observational and pattern recognition skills , 2006, Medical education.

[2]  David Lee Gordon,et al.  Classification of Subtype of Acute Ischemic Stroke: Definitions for Use in a Multicenter Clinical Trial , 1993, Stroke.

[3]  V. Hachinski,et al.  Lack of improvement in patients with acute stroke after treatment with thrombolytic therapy: predictors and association with outcome. , 2004, JAMA.

[4]  L. Goldstein,et al.  Is this patient having a stroke? , 2005, JAMA.

[5]  A. Demchuk,et al.  The iScore predicts efficacy and risk of bleeding in the National Institute of Neurological disorders and Stroke Tissue Plasminogen Activator Stroke Trial. , 2013, Journal of stroke and cerebrovascular diseases : the official journal of National Stroke Association.

[6]  D. Cook,et al.  The value of pretest probability and modified clinical pulmonary infection score to diagnose ventilator-associated pneumonia. , 2008, Journal of critical care.

[7]  P. Clark,et al.  Factors Influencing Stroke Survivors' Quality of Life During Subacute Recovery , 2005, Stroke.

[8]  F. Silver,et al.  Sex Differences in Stroke Care and Outcomes: Results From the Registry of the Canadian Stroke Network , 2005, Stroke.

[9]  H. Diener,et al.  Risk Factors, Outcome, and Treatment in Subtypes of Ischemic Stroke: The German Stroke Data Bank , 2001, Stroke.

[10]  Peter Austin,et al.  The iScore Predicts Effectiveness of Thrombolytic Therapy for Acute Ischemic Stroke , 2012, Stroke.

[11]  Gustavo Saposnik,et al.  IScore: A Risk Score to Predict Death Early After Hospitalization for an Acute Ischemic Stroke , 2011, Circulation.

[12]  V. Hachinski,et al.  Physician knowledge of the benefits, risks, and contraindications of tissue plasminogen activator for acute ischemic stroke. , 2001, Stroke.

[13]  P. Langhorne,et al.  Motor recovery after stroke: a systematic review , 2009, The Lancet Neurology.

[14]  R. Furlan,et al.  San Francisco Syncope Rule, Osservatorio Epidemiologico sulla Sincope nel Lazio risk score, and clinical judgment in the assessment of short-term outcome of syncope. , 2010, The American journal of emergency medicine.

[15]  A. Elstein,et al.  Clinical problem solving and diagnostic decision making: selective review of the cognitive literature , 2002, BMJ : British Medical Journal.

[16]  G. Stranjalis,et al.  A minor revision of Hunt and Hess scale. , 2001, Stroke.

[17]  P. Croskerry.  The Importance of Cognitive Errors in Diagnosis and Strategies to Minimize Them , 2003, Academic medicine : journal of the Association of American Medical Colleges.

[18]  C. McCulloch,et al.  Accuracy of Neurovascular Fellows' Prognostication of Outcome After Subarachnoid Hemorrhage , 2012, Stroke.

[19]  Gustavo Saposnik,et al.  The iScore Predicts Poor Functional Outcomes Early After Hospitalization for an Acute Ischemic Stroke , 2011, Stroke.

[20]  W R Clarke,et al.  Baseline NIH Stroke Scale score strongly predicts outcome after stroke , 1999, Neurology.

[21]  L. Goldman,et al.  The SUPPORT Prognostic Model: Objective Estimates of Survival for Seriously Ill Hospitalized Adults , 1995, Annals of Internal Medicine.

[22]  T. Truelsen,et al.  Comparison of the impact of atrial fibrillation on the risk of stroke and cardiovascular death in women versus men (The Copenhagen City Heart Study). , 2004, The American journal of cardiology.

[23]  H. Mcdonald,et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. , 2005, JAMA.

[24]  K. Stavem,et al.  Derivation and validation of a simple risk score for predicting 1-year mortality in stroke , 2007, Journal of Neurology.