Meta-Analysis: Audit and Feedback Features Impact Effectiveness on Care Quality

Background:Audit and feedback (A&F) has long been used to improve quality of care, albeit with variable results. This meta-analytic study tested whether Feedback Intervention Theory, a framework from industrial/organizational psychology, explains the observed variability in health care A&F research. Method:Data source: studies cited by Jamtvedt's 2006 Cochrane systematic review of A&F, followed by database searches using the Cochrane review's search strategy to identify more recent studies. Inclusion criteria: Cochrane review criteria, plus: presence of a treatment group receiving only A&F; a control group receiving no intervention; a quantitatively measurable outcome; minimum n of 10 per arm; sufficient statistics for effect size calculations. Moderators: presence of discouragement and praise; correct solution, attainment level, velocity, frequency, and normative information; feedback format (verbal, textual, graphic, public, computerized, group vs. individual); goal setting activity. Procedure: meta-analytic procedures using the Hedges-Olkin method. Results:Of 519 studies initially identified, 19 met all inclusion criteria. Studies were most often excluded due to the lack of a feedback-only arm. A&F has a modest, though significant positive effect on quality outcomes (d = 0.40, 95% confidence interval = ±0.20); providing specific suggestions for improvement, written, and more frequent feedback strengthened this effect, whereas graphical and verbal feedback attenuated this effect. Conclusions:A&F effectiveness is improved when feedback is delivered with specific suggestions for improvement, in writing, and frequently. Other feedback characteristics could also potentially improve effectiveness; however, research with stricter experimental controls is needed to identify the specific feedback characteristics that maximize its effectiveness.

[1]  R. Karraker.  Knowledge of results and incorrect recall of plausible multiple-choice alternatives. , 1967 .

[2]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[3]  Steven D. Jones,et al.  Effects of group feedback, goal setting, and incentives on organizational productivity. , 1988 .

[4]  J. Grimshaw,et al.  Effectiveness and efficiency of guideline dissemination and implementation strategies , 2005, International Journal of Technology Assessment in Health Care.

[5]  Gregory E. Prussia,et al.  A covariance structure analysis of employees' response to performance feedback. , 2004, The Journal of applied psychology.

[6]  M. Smith,et al.  Training in Organizations , 1991 .

[7]  M. Mugford,et al.  Effects of feedback of information on clinical practice: a review. , 1991, BMJ.

[8]  E. A. Locke.  Motivational effects of knowledge of results: knowledge or goal setting? , 1967, The Journal of applied psychology.

[9]  Dan Berlowitz,et al.  Provider attitudes toward pay-for-performance programs: development and validation of a measurement instrument. , 2006, Health services research.

[10]  Jacqueline A Pugh,et al.  Audit and feedback and clinical practice guideline adherence: Making feedback actionable , 2006, Implementation science : IS.

[11]  Michael Fordis,et al.  Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. , 2006, JAMA.

[12]  R Baker,et al.  What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review , 2005, BMC health services research.

[13]  Jacqueline A Pugh,et al.  Clinical practice guideline implementation strategy patterns in Veterans Affairs primary care clinics. , 2007, Health services research.

[14]  Göran Tomson,et al.  Effectiveness of feedback for improving case management of malaria, diarrhoea and pneumonia – a randomized controlled trial at provincial hospitals in Lao PDR , 2003, Tropical medicine & international health : TM & IH.

[15]  Martin Eccles,et al.  Implementing clinical guidelines: current evidence and future implications. , 2004, The Journal of continuing education in the health professions.

[16]  Brian S. Mittman,et al.  Changing Provider Behavior , 1992 .

[17]  R. Overton,et al.  A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator , 1998 .

[18]  K. Hood,et al.  General practitioners' use of magnetic resonance imaging: an open randomized trial comparing telephone and written requests and an open randomized controlled trial of different methods of local guideline dissemination. , 2002, Clinical radiology.

[19]  J. Grimshaw,et al.  Evidence-based implementation of evidence-based medicine. , 1999, The Joint Commission journal on quality improvement.

[20]  Chun-Ju Hsiao,et al.  Electronic medical record use by office-based physicians and their practices: United States, 2007. , 2010, National health statistics reports.

[21]  E. Hing,et al.  Electronic medical record use by office-based physicians and their practices: United States, 2006. , 2007, Advance data.

[22]  A. Kluger,et al.  Feedback Interventions , 1998 .

[23]  Laura Petersen,et al.  Does Pay-for-Performance Improve the Quality of Health Care? , 2006, Annals of Internal Medicine.

[24]  Joseph Siemienczuk,et al.  Provider feedback to improve 5A's tobacco cessation in primary care: a cluster randomized clinical trial. , 2007, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[25]  Jeremy M. Grimshaw,et al.  Changing Provider Behavior: An Overview of Systematic Reviews of Interventions , 2001, Medical care.

[26]  A D Oxman,et al.  Audit and feedback: effects on professional practice and health care outcomes. , 2000, The Cochrane database of systematic reviews.

[27]  Anne Whitehead,et al.  Meta-Analysis of Controlled Clinical Trials , 2002 .

[28]  Jacqueline A Pugh,et al.  Not of one mind: mental models of clinical practice guidelines in the Veterans Health Administration. , 2005, Health services research.

[29]  D. Cook,et al.  Evaluating Meta-analyses in the General Surgical Literature: A Critical Appraisal , 2005, Annals of surgery.

[30]  Ingram Olkin,et al.  Clustering estimates of effect magnitude from independent studies , 1983 .

[31]  N. Freemantle,et al.  Audit and feedback versus alternative strategies: effects on professional practice and health care outcomes. , 1998, The Cochrane database of systematic reviews.