Variables that affect the process and outcome of feedback, relevant for medical training: a meta‐review

Feedback is considered important in medical education. The literature is not clear about the mechanisms that contribute to its effects, which are often small to moderate and at times contradictory. A variety of variables seem to influence the impact of feedback on learning. The aim of this study was to determine which variables influence the process and outcomes of feedback in settings relevant to medical education.

[1]  J. Bowen,et al.  Assessing Quality and Costs of Education in the Ambulatory Setting: A Review of the Literature , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[2]  D. Irby,et al.  Teaching in the outpatient clinic , 1997, Journal of General Internal Medicine.

[3]  J. Archer,et al.  Impact of workplace based assessment on doctors’ education and performance: a systematic review , 2010, BMJ : British Medical Journal.

[4]  W. McGaghie,et al.  What is feedback in clinical education? , 2008, Medical education.

[5]  G. Clarebout,et al.  Disentangling instructional roles: the case of teaching and summative assessment , 2011 .

[6]  Mien Segers,et al.  Peer assessment for learning from a social perspective: The influence of interpersonal variables and structural features , 2009 .

[7]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[8]  Helen J. Parkin,et al.  Using technology to encourage student engagement with feedback: a literature review , 2011 .

[9]  Allen I. Huffcutt,et al.  Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. , 1997 .

[10]  P. Roberts,et al.  Peer review in nursing and midwifery: a literature review. , 2007, Journal of clinical nursing.

[11]  J. Gallagher,et al.  Teaching and learning: New models. , 1994 .

[12]  E. Chambers.  An Introduction to Meta-Analysis With Articles From The Journal of Educational Research (1992-2002) , 2004 .

[13]  K. Kraiger,et al.  Study of race effects in objective indices and subjective evaluations of performance: A meta-analysis of performance criteria. , 1986 .

[14]  A D Oxman,et al.  Audit and feedback: effects on professional practice and health care outcomes. , 2006, The Cochrane database of systematic reviews.

[15]  David J. Woehr,et al.  Rater training for performance appraisal: A quantitative review , 1994 .

[16]  Matthew L. Miller,et al.  Feedback Types in Programmed Instruction: A Systematic Review , 2007 .

[17]  Deloris McGee Wanguri.  A Review, an Integration, and a Critique of Cross-Disciplinary Research on Performance Appraisals, Evaluations, and Feedback: 1980-1990: , 1995 .

[18]  D. Irby,et al.  Teaching when time is limited , 2008, BMJ : British Medical Journal.

[19]  K. Frandsen,et al.  On conceptual, theoretical and empirical treatments of feedback in human communication: Fifteen years later , 1993 .

[20]  T. Dornan.  Experience based learning : learning clinical medicine in workplaces , 2006 .

[21]  K. Frandsen,et al.  On conceptual and empirical treatments of feedback in human communication , 1976 .

[22]  Marie Waung,et al.  The Effect of Feedback Packaging on Ratee Reactions1 , 2005 .

[23]  James W. Smither,et al.  DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL, META‐ANALYSIS, AND REVIEW OF EMPIRICAL FINDINGS , 2005 .

[24]  Robert L. Heneman.  The relationship between supervisory ratings and results-oriented measures of performance: A meta-analysis. , 1986 .

[25]  Anders Jonsson,et al.  The use of scoring rubrics: Reliability, validity, and educational consequences , 2007 .

[26]  J. Spark,et al.  Clinical governance: its effect on surgery and the surgeon , 2004, ANZ journal of surgery.

[27]  Cesar G Victora,et al.  How can we achieve and maintain high-quality performance of health workers in low-resource settings? , 2005, The Lancet.

[28]  D. Sharpe,et al.  Of apples and oranges, file drawers and garbage: why validity issues in meta-analysis will not go away. , 1997, Clinical psychology review.

[29]  P. Watzlawick,et al.  Pragmatics of Human Communication: A Study of Interactional Patterns, Pathologies and Paradoxes , 1964 .

[30]  Ralph Gonzales,et al.  Improving Antibiotic Selection: A Systematic Review and Quantitative Analysis of Quality Improvement Strategies , 2006, Medical care.

[31]  Sue Margaret Norton,et al.  Peer assessments of performance and ability: An exploratory meta-analysis of statistical artifacts and contextual moderators , 1992 .

[32]  D. R. Ilgen,et al.  Performance Appraisal Process Research in the 1980s: What Has It Contributed to Appraisals in Use? , 1993 .

[33]  R. Hogarth,et al.  Determinants of Linear Judgment: A Meta-Analysis of Lens Model Studies , 2007, Psychological bulletin.

[34]  Harold D. Lasswell.  Describing the Contents of Communications , 1946 .

[35]  D M Irby,et al.  Teaching and learning in ambulatory care settings: a thematic review of the literature , 1995, Academic medicine : journal of the Association of American Medical Colleges.

[36]  Jon Veloski,et al.  Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7 , 2006, Medical teacher.

[37]  Chockalingam Viswesvaran,et al.  Comparative analysis of the reliability of job performance ratings , 1996 .

[38]  Deborah DiazGranados,et al.  Author Notes , 1994, Schools of Thought.

[39]  M. Taylor,et al.  Consequences of individual feedback on behavior in organizations. , 1979 .

[40]  K. Kraiger,et al.  The Relation of Job Knowledge, Job Performance, and Supervisory Ratings as a Function of Ratee Race , 1990 .

[41]  D. Fenner.  Avoiding pitfalls: lessons in surgical teaching. , 2006, Obstetrics and gynecology clinics of North America.

[42]  J. Pratt,et al.  Temporal Locus of Knowledge of Results: A Meta-Analytic Review , 1995, Perceptual and motor skills.

[43]  Alessandro Liberati,et al.  Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study , 2005, BMJ : British Medical Journal.

[44]  Vernon C. Hall,et al.  The overjustification effect: A meta‐analysis , 1995 .

[45]  Vivien Coates,et al.  Performance assessment in health care providers: a critical review of evidence and current practice. , 2007, Journal of nursing management.

[46]  Donald B. Fedor,et al.  Individual differences in feedback propensities and training performance , 2003 .

[47]  J. R. Larson.  The performance feedback process: A preliminary model , 1984 .

[48]  Shaofeng Li.  The Effectiveness of Corrective Feedback in SLA: A Meta-Analysis. , 2010 .

[49]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[50]  Roy B. Clariana,et al.  Multiple-Try Feedback and Higher-Order Learning Outcomes. , 2005 .

[51]  Chiungjung Huang.  Magnitude of Task-Sampling Variability in Performance Assessment: A Meta-Analysis , 2009 .

[52]  P. Bower,et al.  Can the feedback of patient assessments, brief training, or their combination, improve the interpersonal skills of primary care physicians? A systematic review , 2008, BMC health services research.

[53]  Neal M. Kingston,et al.  Formative Assessment: A Meta-Analysis and a Call for Research , 2011 .

[54]  John Schaubroeck,et al.  A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. , 1988 .

[55]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[56]  M. S. Patel,et al.  An introduction to meta-analysis. , 1989, Health Policy.

[57]  Mark Albanese,et al.  Systematic Reviews: What Are They and Why Should We Care? , 2002, Advances in health sciences education : theory and practice.

[58]  F. Sanford,et al.  Speech and personality. , 1942 .

[59]  M. Doherty,et al.  Effects of cognitive feedback on performance. , 1989 .

[60]  V. Shute.  Focus on Formative Feedback , 2007 .

[61]  R. Ammons.  Propaganda, communication, and public opinion; a comprehensive reference guide. , 1947 .

[62]  D. Irby,et al.  Teaching in the outpatient clinic. Practical tips. , 1997, Journal of general internal medicine.

[63]  Amy M. Knight,et al.  Physician characteristics associated with proficiency in feedback skills , 2006, Journal of General Internal Medicine.

[64]  E. Morrison,et al.  Feedback Seeking Behavior: A Review of the Literature and Implications for HRD Practitioners [and] Invited Reaction: Comments on Madzar's Article. , 1995 .

[65]  Lyman Bryson.  The communication of ideas : a series of addresses , 1948 .

[66]  George P. Richardson,et al.  Feedback Thought in Social Science and Systems Theory , 1991 .

[67]  A. Harden,et al.  Methodological Issues in Combining Diverse Study Types in Systematic Reviews , 2005 .

[68]  L. P. Cusella,et al.  Feedback, motivation, and performance. , 1987 .

[69]  J. Norcini,et al.  Workplace-based assessment as an educational tool: AMEE Guide No. 31 , 2007, Medical teacher.

[70]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[71]  David E. Smith,et al.  Training Programs for Performance Appraisal: A Review , 1986 .

[72]  William H. Bommer,et al.  ON THE INTERCHANGEABILITY OF OBJECTIVE AND SUBJECTIVE MEASURES OF EMPLOYEE PERFORMANCE: A META-ANALYSIS , 1995 .

[73]  D. Briscoe,et al.  Employee Performance Management Across Borders: A Review of Relevant Academic Literature , 2009 .

[74]  R. Bruce McAfee,et al.  The use of incentives/feedback to enhance work place safety: A critique of the literature. , 1989 .

[75]  D. Sadler.  Formative assessment and the design of instructional systems , 1989 .

[76]  H. Lasswell.  The structure and function of communication in society , 2007 .

[77]  Regina Vollmeyer,et al.  Motivational Effects on Self-Regulated Learning with Different Tasks , 2006 .

[78]  R. Fukkink,et al.  Video Feedback in Education and Training: Putting Learning in the Picture , 2011 .

[79]  Chen-Lin C. Kulik,et al.  Timing of Feedback and Verbal Learning , 1988 .

[80]  Chen-Lin C. Kulik,et al.  The Instructional Effect of Feedback in Test-Like Events , 1991 .

[81]  J. Avorn,et al.  Improving drug prescribing in primary care: a critical analysis of the experimental literature. , 1989, The Milbank quarterly.

[82]  Jane M. Young,et al.  Audit and feedback: effects on professional practice and healthcare outcomes. , 2012, The Cochrane database of systematic reviews.

[83]  I. M. Jawahar,et al.  WHERE ALL THE CHILDREN ARE ABOVE AVERAGE: THE PERFORMANCE APPRAISAL PURPOSE EFFECT , 1997 .

[84]  Marina Schroder.  Handbook Of Organizational Communication An Interdisciplinary Perspective , 2016 .

[85]  Adrian Furnham,et al.  Accepting personality test feedback: A review of the Barnum effect , 1987 .

[86]  R. Lyster,et al.  ORAL FEEDBACK IN CLASSROOM SLA , 2010, Studies in Second Language Acquisition.

[87]  Roger Azevedo,et al.  A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction , 1995 .

[88]  E. Mory.  Feedback research revisited. , 2004 .

[89]  K. Murphy,et al.  Evaluating the Performance of Paper People , 1986 .

[90]  S. Craig.  Direct observation of clinical practice in emergency medicine education. , 2011, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[91]  M. Delgado-Rodríguez.  Systematic reviews of meta-analyses: applications and limitations , 2006, Journal of Epidemiology and Community Health.