Consensus Methods to Identify a Set of Potential Performance Indicators for Systems of Emergency and Urgent Care

Objectives To identify a comprehensive set of indicators to enable Primary Care Trust (PCT) commissioners in England and other NHS decision-makers to monitor the performance of systems of emergency and urgent care for which they are responsible. Methods Using a combination of Delphi RAND methods in three successive rounds of consultation and nominal group review, we canvassed expert opinion on 70 potential indicators as good measures of system performance. The two Delphi panels consisted of senior clinicians and researchers, and urgent care leads and commissioners in PCTs and Strategic Health Authorities (SHAs). The indicators were formatted into a questionnaire according to whether they were outcome, process, structure, or equity-based measures. Participants scored each indicator on a Likert scale of 1-9 and had the opportunity to consider their scores informed by the group scores and feedback. The questionnaire was refined after each round. To ensure that the indicators rated most highly by the Delphi panels covered all dimensions of performance, the results of the Delphi were reviewed by a nominal group consisting of two researchers and three clinicians from the local health services research network (LHSR). Results Overall, the process yielded 16 candidate indicators. It also produced a core set of serious, emergency and urgent care-sensitive conditions (defined as conditions whose exacerbations should be managed by a well-performing system without admission to an inpatient bed), for use with the indicators. Conclusions System-wide measures to monitor performance across multiple services should encourage providers to work for patient benefit in an integrated way. They will also assist commissioners to monitor and improve emergency and urgent care for their local populations. The indicators are now being calculated using routinely available data, and tested for their responsiveness to capture change over time.

[1]  Mike Pringle,et al.  Measuring “goodness” in individuals and healthcare systems , 2002, BMJ : British Medical Journal.

[2]  B. Burnand,et al.  The RAND/UCLA Appropriateness Method User's Manual , 2001 .

[3]  M. Boers,et al.  Rationale and design of The Delphi Trial – I(RCT)2: international randomized clinical trial of rheumatoid craniocervical treatment, an intervention-prognostic trial comparing 'early' surgery with conservative treatment [ISRCTN65076841] , 2006, BMC musculoskeletal disorders.

[4]  A. Jensen,et al.  Randomised controlled trial of a shared care programme for newly referred cancer patients: bridging the gap between general practice and hospital , 2003, Quality & safety in health care.

[5]  Chia-Chien Hsu,et al.  The Delphi Technique: Making Sense of Consensus , 2007 .

[6]  Apho.  The good indicators guide: understanding how to use and choose indicators , 2008 .

[7]  Samuel DiGangi,et al.  Assessing unidimensionality: A comparison of Rasch modeling, Parallel analysis, and TETRAD , 2007 .

[8]  H. Hearnshaw,et al.  Left in limbo: patients' views on care across the primary/secondary interface. , 1999, Quality in health care : QHC.

[9]  A. O’Cathain,et al.  Characteristics of the Emergency and Urgent Care System Important to Patients: A Qualitative Study , 2008, Journal of health services research & policy.

[10]  K. Mackway-Jones,et al.  A Delphi study to identify performance indicators for emergency medicine , 2004, Emergency Medicine Journal.

[11]  J. Ayres,et al.  Defining and investigating difficult asthma: developing quality indicators. , 2006, Respiratory medicine.

[12]  S. Mason,et al.  Targets and moving goal posts: changes in waiting times in a UK emergency department , 2005, Emergency Medicine Journal.

[13]  S. Bird,et al.  Performance indicators: good, bad, and ugly , 2004 .

[14]  D. Redelmeier,et al.  Mortality among patients admitted to hospitals on weekends as compared with weekdays. , 2001, The New England journal of medicine.

[15]  L Price,et al.  Treating the clock and not the patient: ambulance response times and risk , 2006, Quality and Safety in Health Care.

[16]  MURRAY TUROFF,et al.  Philosophical and Methodological Foundations of Delphi , 2002 .

[17]  L. E. Peterson,et al.  Eliciting expert opinion using the Delphi technique: identifying performance indicators for cardiovascular disease. , 1998, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[18]  C. Sanderson,et al.  Conditions for Which Onset or Hospital Admission is Potentially Preventable by Timely and Effective Ambulatory Care , 2000, Journal of health services research & policy.

[19]  An integrated care pathway for menorrhagia across the primary–secondary interface: patients’ experience, clinical outcomes, and service utilisation , 2007, Quality and Safety in Health Care.