Effects of Feedback in a Computer-Based Learning Environment on Students’ Learning Outcomes

In this meta-analysis, we investigated the effects of methods for providing item-based feedback in a computer-based environment on students’ learning outcomes. From 40 studies, 70 effect sizes were computed, which ranged from −0.78 to 2.29. A mixed model was used for the data analysis. The results show that elaborated feedback (EF; e.g., providing an explanation) produced larger effect sizes (0.49) than feedback regarding the correctness of the answer (KR; 0.05) or providing the correct answer (KCR; 0.32). EF was particularly more effective than KR and KCR for higher order learning outcomes. Effect sizes were positively affected by EF feedback, and larger effect sizes were found for mathematics compared with social sciences, science, and languages. Effect sizes were negatively affected by delayed feedback timing and by primary and high school. Although the results suggested that immediate feedback was more effective for lower order learning than delayed feedback and vice versa, no significant interaction was found.

[1]  K. Morgan‐Short,et al.  Positive Evidence Versus Explicit Rule Presentation and Explicit Negative Feedback: A Computer-Assisted Study. , 2004 .

[2]  Philip Murphy,et al.  Click here Terms of use : Click here Web-based collaborative reading exercises for learners in remote locations : the effects of computer-mediated feedback and interaction via computer-mediated communication , 2010 .

[3]  R. W. Kulhavy.  Feedback in Written Instruction , 1977 .

[4]  D. Wiliam.  What Is Assessment for Learning , 2011 .

[5]  Erin Marie Furtak,et al.  Meta‐analytic methodology and inferences about the efficacy of formative assessment , 2012 .

[6]  Keith A. Hall,et al.  Gradient- and Full-Response Feedback In Computer-Assisted Instruction , 1968 .

[7]  Francis M. Dwyer,et al.  The effect of questions and feedback used to complement static and animated visualization on tests measuring different educational objectives , 2006 .

[8]  Howard D. White,et al.  SCIENTIFIC COMMUNICATION AND LITERATURE RETRIEVAL , 2019, The Handbook of Research Synthesis and Meta-Analysis.

[9]  Theodorus Johannes Hendrikus Maria Eggen,et al.  The effectiveness of methods for providing written feedback through a computer-based assessment for learning: a systematic review , 2011 .

[10]  Jeffrey K. Smith,et al.  Effects of differential feedback on students' examination performance. , 2009, Journal of experimental psychology. Applied.

[11]  William R. Shadish,et al.  Combining estimates of effect size. , 1994 .

[12]  R. Orwin.  A fail-safe N for effect size in meta-analysis. , 1983 .

[13]  Alfred Valdez,et al.  Encouraging mindful feedback processing: computer-based instruction in descriptive statistics , 2008 .

[14]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[15]  Tamara van Gog,et al.  Content and timing of feedback in a web-based learning environment: effects on learning as a function of prior knowledge , 2008, Interact. Learn. Environ..

[16]  Michael C. Rodriguez.  The Role of Classroom Assessment in Student Performance on TIMSS , 2004 .

[17]  S. Narciss.  Feedback Strategies for Interactive Learning Tasks , 2007 .

[18]  S. Wager.  The effect of immediacy and type of informative feedback on retention in a computer-assisted task , 1983 .

[19]  Harold F. O'Neil,et al.  The role of task-specific adapted knowledge of response feedback in algebra problem solving online homework in a college remedial course , 2008 .

[20]  P. Smith,et al.  Instructional Design (3rd ed. , 2005 .

[21]  John Hattie,et al.  INSTRUCTION BASED ON FEEDBACK , 2010 .

[22]  Nina Moreno.  The effects of type of task and type of feedback on L2 development in CALL , 2007 .

[23]  Wim J. Nijhof,et al.  Effects of complex feedback on computer-assisted modular instruction , 2002, Comput. Educ..

[24]  R. Rosenthal.  Parametric measures of effect size. , 1994 .

[25]  D. Carnine,et al.  Elaborated Corrective Feedback and the Acquisition of Reasoning Skills: A Study of Computer-Assisted Instruction , 1987, Exceptional children.

[26]  Robert G. Orwin,et al.  A Fail-SafeN for Effect Size in Meta-Analysis , 1983 .

[27]  Walter P. Vispoel,et al.  Psychometric Characteristics of Computer-Adaptive and Self-Adaptive Vocabulary Tests: The Role of Answer Feedback and Test Anxiety. , 1998 .

[28]  Susanne Narciss,et al.  Fostering achievement and motivation with bug-related tutoring feedback in a computer-based training for written subtraction. , 2006 .

[29]  Ronald P. Leow,et al.  Computerized Task-Based Exposure, Explicitness, Type of Feedback, and Spanish L2 Development , 2004 .

[30]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[31]  Noriko Nagata,et al.  Intelligent Computer Feedback for Second Language Instruction , 1993 .

[32]  R. W. Kulhavy,et al.  Feedback in written instruction: The place of response certitude , 1989 .

[33]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[34]  V. Shute.  Focus on Formative Feedback , 2007 .

[35]  P. Black,et al.  Inside the Black Box: Raising Standards through Classroom Assessment , 2010 .

[36]  James D. Klein,et al.  Control of Practice and Level of Feedback in Computer-Based Instruction , 1995 .

[37]  Francis M. Dwyer,et al.  The Effect of Online Gaming, Cognition and Feedback Type in Facilitating Delayed Achievement of Different Learning Objectives , 2005 .

[38]  Mine Munyofu.  Effects of varied enhancement strategies (chunking, feedback, gaming) in complementing animated instruction in facilitating different types of learning objectives , 2008 .

[39]  Liesbeth Kester,et al.  Dynamic task selection: Effects of feedback and learner control on efficiency and motivation , 2009 .

[40]  Noriko Nagata,et al.  A Study of Consciousness-Raising by Computer: The Effect of Metalinguistic Feedback on Second Language Learning , 1995 .

[41]  Bernard P. Veldkamp,et al.  Attention paid to feedback provided by a computer-based assessment for learning on information literacy , 2011, Comput. Educ..

[42]  Ingram Olkin,et al.  Estimation of a Single Effect Size: Parametric and Nonparametric Methods , 1985 .

[43]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[44]  John Merrill,et al.  Levels of questioning and forms of feedback: instructional factoars in courseware design , 1987 .

[45]  John Hattie,et al.  Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement , 2008 .

[46]  V. Kopp,et al.  Fostering diagnostic knowledge through computer‐supported, case‐based worked examples: effects of erroneous examples and feedback , 2008, Medical education.

[47]  Joanna Dunlap,et al.  Identifying the relationship between feedback provided in computer-assisted instructional modules, science self-efficacy, and academic achievement , 2006 .

[48]  Gary R. Morrison,et al.  The Effects of Feedback and Incentives on Achievement in Computer-Based Instruction , 1995 .

[49]  Susanne Narciss,et al.  How to design informative tutoring feedback for multi-media learning , 2004 .

[50]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[51]  Orit Zeichner,et al.  Using Technology to Enhance Mathematical Reasoning: Effects of Feedback and Self-Regulation Learning , 2001 .

[52]  E. Mory.  Feedback research revisited. , 2004 .

[53]  Sandra T. Miller.  Formative Computer-based Assessments: The potentials and pitfalls of two formative computer-based assessments used in professional learning programs , 2009 .

[54]  Wim van den Noortgate,et al.  Adaptive item-based learning environments based on the item response theory: possibilities and challenges , 2010, J. Comput. Assist. Learn..

[55]  Roger Azevedo,et al.  A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction , 1995 .

[56]  Gordon Stobart,et al.  Testing Times: The Uses and Abuses of Assessment , 2008 .

[57]  Bernard P. Veldkamp,et al.  Effects of feedback in a computer-based assessment for learning , 2012, Comput. Educ..

[58]  F J Papa,et al.  The effects of immediate online feedback upon diagnostic performance. , 1999, Academic medicine : journal of the Association of American Medical Colleges.

[59]  Mengqiao Xu.  An investigation of the effectiveness of intelligent elaborative feedback afforded by pedagogical agents on improving young Chinese language learners' vocabulary acquisition , 2009 .

[60]  Neal M. Kingston,et al.  Formative Assessment: A Meta-Analysis and a Call for Research , 2011 .

[61]  Philip Murphy,et al.  READING COMPREHENSION EXERCISES ONLINE: THE EFFECTS OF FEEDBACK, PROFICIENCY AND INTERACTION Paginated PDF Version , 2007 .

[62]  James D. Klein,et al.  Control of feedback in computer-assisted instruction , 1991 .

[63]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[64]  L. Hedges.  Estimation of effect size from a series of independent experiments. , 1982 .

[65]  R. Moreno.  Decreasing Cognitive Load for Novice Students: Effects of Explanatory versus Corrective Feedback in Discovery-Based Multimedia , 2004 .

[66]  Chen-Lin C. Kulik,et al.  The Instructional Effect of Feedback in Test-Like Events , 1991 .

[67]  R. Moreno,et al.  Cognitive load and learning effects of having students organize pictures and words in multimedia environments: The role of student interactivity and feedback , 2005 .

[68]  Helmer Strik,et al.  The effectiveness of computer-based speech corrective feedback for improving segmental quality in L2 Dutch , 2008, ReCALL.

[69]  R. Clariana,et al.  The effects of recognition and recall study tasks with feedback in a computer-based vocabulary lesson , 2001 .

[70]  Bernard P. Veldkamp,et al.  Computerized adaptive testing in computer assisted learning , 2011 .

[71]  Hans Cuypers,et al.  Computer-based feedback in linear algebra: Effects on transfer performance and motivation , 2010, Comput. Educ..

[72]  D. Sadler.  Formative assessment and the design of instructional systems , 1989 .

[73]  R. Bennett.  Formative assessment: a critical review , 2011 .

[74]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[75]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[76]  Steven L. Wise,et al.  The Role of Item Feedback in Self-Adapted Testing , 1997 .

[77]  J. H. McMillan,et al.  Studies of the Effect of Formative Assessment on Student Achievement: So Much More Is Needed. , 2013 .

[78]  Dirk Ifenthaler,et al.  Bridging the gap between expert-novice differences : the model-based feedback approach , 2009, CELDA 2009.

[79]  Matthew L. Miller,et al.  Feedback Types in Programmed Instruction: A Systematic Review , 2007 .

[80]  Barbara L. Grabowski,et al.  Improving self-regulation, learning strategy use, and achievement with metacognitive feedback , 2010 .

[81]  L. Hedges.  Distribution Theory for Glass's Estimator of Effect size and Related Estimators , 1981 .