The level of evidence for emergency department performance indicators: systematic review

The aim of this study was to perform a comprehensive systematic review of emergency department performance indicators in relation to evidence. A systematic search was performed through PUBMED, EMBASE, CINAHL and COCHRANE databases with (and including synonyms of) the search words: [emergency medicine OR emergency department] AND [quality indicator(s) OR performance indicator(s) OR performance measure(s)]. Articles were included according to the inclusion/exclusion criteria using the PRISMA protocol. The level of evidence was rated according to the evidence levels by the Oxford Centre for Evidence-Based Medicine. Performance indicators were extracted and organized into five categories; outcome, process, satisfaction, equity and structural/organizational measures. Six thousand four hundred and forty articles were initially identified; 127 provided evidence for/against a minimum of one performance indicator: these were included for further study. Of the 127 articles included, 113 (92%) were primary research studies and only nine (8%) were systematic reviews. Within the 127 articles, we found evidence for 202 individual indicators. Approximately half (n=104) of all this evidence (n=202) studied process-type indicators. Only seven articles (6%) qualified for high quality (level 1b). Sixty-six articles (51%) were good retrospective quality (level 2b or better), whereas the remaining articles were either intermediate quality (25% level 3a or 3b) or poor quality (17% level 4 or 5). We found limited evidence for most emergency department performance indicators, with the majority presenting a low level of evidence. Thus, a core group of evidence-based performance indicators cannot currently be recommended on the basis of this broad review of the literature.

[1]  E. Boudreaux,et al.  The use of performance improvement methods to enhance emergency department patient satisfaction in the United States: a critical review of the literature and suggestions for future research. , 2006, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[2]  Oxford Centre for Evidence-based Medicine Levels of Evidence (January 2001) , 2014 .

[3]  L. Lind,et al.  A systematic review of triage-related interventions to improve patient flow in emergency departments , 2011, Scandinavian journal of trauma, resuscitation and emergency medicine.

[4]  Jesse M Pines,et al.  A field test of time-based emergency department quality measures. , 2012, Annals of emergency medicine.

[5]  Fiona Lecky,et al.  The International Federation for Emergency Medicine framework for quality and safety in the emergency department , 2013, Emergency Medicine Journal.

[6]  Anne Wilson,et al.  The clinical effectiveness of nurse practitioners' management of minor injuries in an adult emergency department: a systematic review. , 2009, International journal of evidence-based healthcare.

[7]  David McD Taylor,et al.  Review article: Leaving the emergency department without being seen , 2008, Emergency medicine Australasia : EMA.

[8]  M. Mccarthy,et al.  Measures of crowding in the emergency department: a systematic review. , 2011, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[9]  C. Camargo,et al.  Factors Associated With Longer Emergency Department Length of Stay for Children With Bronchiolitis: A Prospective Multicenter Study , 2009, Pediatric emergency care.

[10]  A. Clarey,et al.  Patients who leave emergency departments without being seen: literature review and English data analysis , 2011, Emergency Medicine Journal.

[11]  O A Arah,et al.  Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[12]  Papaarangi Reid,et al.  Selection and validation of quality indicators for the Shorter Stays in Emergency Departments National Research Project , 2012, Emergency medicine Australasia : EMA.

[13]  Shari J. Welch,et al.  Emergency department operational metrics, measures and definitions: results of the Second Performance Measures and Benchmarking Summit. , 2011, Annals of emergency medicine.

[14]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[15]  E. Weber,et al.  Implications of England's four-hour target for quality of care and resource use in the emergency department. , 2012, Annals of emergency medicine.

[16]  Peter Jacobsen,et al.  Evaluation of emergency department performance – a systematic review on recommended performance and quality-in-care measures , 2013, Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine.

[17]  Jeremiah D Schuur,et al.  The role of the Society for Academic Emergency Medicine in the development of guidelines and performance measures. , 2010, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[18]  D. Aronsky,et al.  Systematic review of emergency department crowding: causes, effects, and solutions. , 2008, Annals of emergency medicine.

[19]  James J. Yang,et al.  Measuring Emergency Physicians' Work: Factoring in Clinical Hours, Patients Seen, and Relative Value Units into 1 Metric , 2012, The western journal of emergency medicine.

[20]  Daniel J Pallin,et al.  Parametric versus nonparametric statistical tests: the length of stay example. , 2010, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[21]  G. Innes,et al.  Prospective time study derivation of emergency physician workload predictors. , 2005, CJEM.

[22]  J. Mainz.  Defining and classifying clinical indicators for quality improvement. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[23]  J. Pines,et al.  The measurement of time to first antibiotic dose for pneumonia in the emergency department: a white paper and position statement prepared for the American Academy of Emergency Medicine. , 2009, The Journal of emergency medicine.

[24]  Stuart Lewena,et al.  Leaving the paediatric emergency department without being seen: Understanding the patient and the risks , 2012, Journal of paediatrics and child health.

[25]  Brian H Rowe,et al.  Prioritizing performance measurement for emergency department care: consensus on evidence-based quality of care indicators. , 2011, CJEM.

[26]  Jean Carletta,et al.  Assessing Agreement on Classification Tasks: The Kappa Statistic , 1996, CL.

[27]  P. Cameron,et al.  Improving timeliness while improving the quality of emergency department care , 2012, Emergency Medicine Australasia.

[28]  P. Layde,et al.  Health-related quality of life in pediatric minor injury: reliability, validity, and responsiveness of the Pediatric Quality of Life Inventory in the emergency department. , 2012, Archives of pediatrics & adolescent medicine.

[29]  S. Thomas,et al.  Ongoing visual analog score display improves Emergency Department pain care. , 2004, The Journal of emergency medicine.

[30]  L. Leibovici,et al.  Time to first antibiotic dose for patients hospitalised with community-acquired pneumonia. , 2013, International journal of antimicrobial agents.

[31]  J. Pines,et al.  Exogenous predictors of national performance measures for emergency department crowding. , 2012, Annals of emergency medicine.

[32]  B. Vandermeer,et al.  The role of triage nurse ordering on mitigating overcrowding in emergency departments: a systematic review. , 2011, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[33]  Todd G Nick,et al.  Comparison of the National Emergency Department Overcrowding Scale and the Emergency Department Work Index for quantifying emergency department crowding. , 2006, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[34]  Nathan P. Charlton,et al.  Accuracy of Internet Recommendations for Prehospital Care of Venomous Snake Bites , 2010, Wilderness & environmental medicine.

[35]  Shari J. Welch,et al.  Emergency department performance measures and benchmarking summit. , 2006, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[36]  C. Camargo,et al.  ED visit volume and quality of care in acute exacerbations of chronic obstructive pulmonary disease. , 2009, The American journal of emergency medicine.

[37]  Peter Kuster,et al.  Handbook Of Practical Program Evaluation , 2016 .

[38]  J. Hedges,et al.  Patient satisfaction investigations and the emergency department: what does the literature say? , 2000, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[39]  Sharon E Straus,et al.  Evidence for quality indicators to evaluate adult trauma care: A systematic review* , 2011, Critical care medicine.