Effects of multiple-try feedback and question type during mathematics problem solving on performance in similar problems

In a study of mathematics problem solving, the effect of providing multiple-try feedback on later success in solving similar problems was examined. Participants solved mathematics problems that were presented as either multiple-choice or open-ended questions, and were provided with one of four types of feedback: no feedback (NF), immediate knowledge of the correct response (KCR), multiple-try feedback with knowledge of the correct response (MTC), or multiple-try feedback with hints after an initial incorrect response (MTH). Results showed that gains in performance were larger in the open-ended than multiple-choice condition. Furthermore, gains under NF and KCR were similar, gains were larger under MTC than KCR, and gains were larger under MTH than MTC. The implications of these results for the design of assessments for learning are discussed. Transfer effect across math problem types and feedback conditions was examined.Solving open-ended problems resulted in larger transfer than multiple-choice.No feedback showed similar transfer effect as single try feedback.Multiple try feedback showed larger transfer effect than single try feedback.Hint feedback showed larger transfer effect than multiple try feedback.

[1]  V. Shute.  Focus on Formative Feedback , 2008 .

[2]  Roger Azevedo,et al.  A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction , 1995 .

[3]  Jeffrey D. Karpicke,et al.  Retrieval practice with short-answer, multiple-choice, and hybrid tests , 2014, Memory.

[4]  Raymond W. Kulhavy,et al.  Feedback complexity and corrective efficiency , 1985 .

[5]  Tamara van Gog,et al.  Content and timing of feedback in a web-based learning environment: effects on learning as a function of prior knowledge , 2008, Interact. Learn. Environ..

[6]  S. Pressey.  Development and Appraisal of Devices Providing Immediate Automatic Scoring of Objective Tests and Concomitant Self-Instruction , 1950 .

[7]  Chen-Lin C. Kulik,et al.  The Instructional Effect of Feedback in Test-Like Events , 1991 .

[8]  G. Salomon,et al.  Skill may not be enough: The role of mindfulness in learning and transfer , 1987 .

[9]  D. Gilman.  Comparison of Several Feedback Methods for Correcting Errors by Computer-Assisted Instruction. , 1969 .

[10]  R. Clariana,et al.  The effects of recognition and recall study tasks with feedback in a computer-based vocabulary lesson , 2001 .

[11]  R. Bennett.  Formative assessment: a critical review , 2011 .

[12]  M. Wittrock.  Generative Learning Processes of the Brain , 1992 .

[13]  Michael C. Rodriguez.  The Role of Classroom Assessment in Student Performance on TIMSS , 2004 .

[14]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[15]  Mark R. Lepper,et al.  The wisdom of practice: Lessons learned from the study of highly effective tutors. , 2002 .

[16]  Heather H. Mitchell,et al.  AutoTutor: A tutor with dialogue in natural language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[17]  P. Black,et al.  Inside the Black Box: Raising Standards through Classroom Assessment , 2010 .

[18]  Matthew L. Miller,et al.  Feedback Types in Programmed Instruction: A Systematic Review , 2007 .

[19]  Michael A. McDaniel,et al.  The generation effect: A meta-analytic review , 2007, Memory & cognition.

[20]  N. J. Slamecka,et al.  The Generation Effect: Delineation of a Phenomenon , 1978 .

[21]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[22]  R. W. Kulhavy,et al.  Feedback in written instruction: The place of response certitude , 1989 .

[23]  Roy B. Clariana,et al.  Multiple-Try Feedback and Higher-Order Learning Outcomes. , 2005 .

[24]  Elizabeth J Marsh,et al.  The positive and negative consequences of multiple-choice testing. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[25]  Leonard S. Feldt,et al.  A test of the hypothesis that cronbach's alpha or kuder-richardson coefficent twenty is the same for two tests , 1969 .

[26]  Michael E. Martinez.  Cognition and the question of test item format , 1999 .

[27]  A. C. Butler,et al.  Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing , 2008, Memory & cognition.

[28]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[29]  Arthur C. Graesser,et al.  AutoTutor: A simulation of a human tutor , 1999, Cognitive Systems Research.

[30]  Andrew C. Butler,et al.  Explanation Feedback Is Better Than Correct Answer Feedback for Promoting Transfer of Learning , 2013 .

[31]  Henry L. Roediger,et al.  Test format and corrective feedback modify the effect of testing on long-term retention , 2007 .

[32]  Elizabeth Stone,et al.  Differences in Reaction to Immediate Feedback and Opportunity to Revise Answers for Multiple-Choice and Open-Ended Questions , 2016, Educational and psychological measurement.

[33]  Andrew S. Gibbons,et al.  Computer-Based Instruction: Design and Development , 1998 .

[34]  John R. Anderson,et al.  Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes , 2001, CHI.

[35]  Leonard S. Feldt,et al.  A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample , 1980 .