Feedback reduces the metacognitive benefit of tests.

Testing long-term memory has dual benefits: It enhances learning and it helps learners discriminate what they know from what they do not know. The latter benefit, known as delayed judgment of learning (dJOL) effect, has been well documented, but in prior research participants have not been provided with test feedback. Yet when people study they almost universally (a) get feedback and (b) judge their learning subsequent to receiving the feedback. Thus, in the first three experiments, participants made JOLs following tests with feedback. Adding feedback significantly decreased the dJOL effect relative to conditions taking a test without receiving feedback. In Experiment 4, participants made decisions about which items to restudy (without actually restudying); adding feedback also decreased the accuracy of these decisions. These findings suggest that, in realistic situations, tests enhance self-monitoring, but not as much as previously thought. Judging memory based on prior test performance and ignoring the effects of feedback appears to produce an "illusion of not knowing."

[1]  Bridgid Finn,et al.  Framing effects on metacognitive monitoring and control , 2008, Memory & cognition.

[2]  T. O. Nelson,et al.  A comparison of current measures of the accuracy of feeling-of-knowing predictions. , 1984, Psychological bulletin.

[3]  R. Bjork,et al.  Unsuccessful retrieval attempts enhance subsequent learning. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[4]  Jeffrey D. Karpicke,et al.  The Power of Testing Memory Basic Research and Implications for Educational Practice , 2006 .

[5]  Nate Kornell,et al.  Optimising self-regulated study: The benefits—and costs—of dropping flashcards , 2008, Memory.

[6]  A. Glenberg,et al.  The illusion of knowing: Failure in the self-assessment of comprehension , 1982 .

[7]  Nate Kornell,et al.  Study efficacy and the region of proximal learning framework. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Nate Kornell,et al.  A stability bias in human memory: overestimating remembering and underestimating learning. , 2009, Journal of experimental psychology. General.

[9]  Janet Metcalfe,et al.  Judgments of Learning are Influenced by Memory for Past Test. , 2008, Journal of memory and language.

[10]  Jon Sprouse.  A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory , 2010, Behavior research methods.

[11]  Matthew Jensen Hays,et al.  The costs and benefits of providing feedback during learning , 2010, Psychonomic bulletin & review.

[12]  Janet Metcalfe,et al.  The role of memory for past test in the underconfidence with practice effect. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[13]  D. Nowlan,et al.  Costs and benefits. , 1980, Irish medical journal.

[14]  J. Metcalfe,et al.  Metacognition : knowing about knowing , 1994 .

[15]  Matthew G. Rhodes,et al.  Monitoring memory errors: The influence of the veracity of retrieved information on the accuracy of judgements of learning , 2011, Memory.

[16]  R. Bjork,et al.  Self-regulated Study 1 Running Head : SELF-REGULATED STUDY Optimizing self-regulated study : The benefits — and costs — of dropping flashcards , 2012 .

[17]  Barbara A. Spellman,et al.  When Predictions Create Reality: Judgments of Learning May Alter What They Are Intended to Assess , 1992 .

[18]  Colin M. Macleod,et al.  The production effect: delineation of a phenomenon. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[19]  T. O. Nelson.  Consciousness and metacognition. , 1996 .

[20]  Asher Koriat,et al.  The intricate relationships between monitoring and control in metacognition: lessons for the cause-and-effect relation between subjective experience and behavior. , 2006, Journal of experimental psychology. General.

[21]  John Dunlosky,et al.  Improving college students’ evaluation of text learning using idea-unit standards , 2011, Quarterly journal of experimental psychology.

[22]  T. O. Nelson.  Metamemory: A Theoretical Framework and New Findings , 1990 .

[23]  Fredrik U. Jönsson,et al.  Study for now, but judge for later: delayed judgments of learning promote long-term retention. , 2012, Scandinavian journal of psychology.

[24]  Matthew G. Rhodes,et al.  The influence of delaying judgments of learning on metacognitive accuracy: a meta-analytic review. , 2011, Psychological bulletin.

[25]  Nate Kornell,et al.  The pretesting effect: do unsuccessful retrieval attempts enhance learning? , 2009, Journal of experimental psychology. Applied.

[26]  H. Pashler,et al.  When does feedback facilitate learning of words? , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[27]  S. M. Nelson,et al.  The Testing Effect , 2014 .

[28]  Karen Swan,et al.  Using standards to improve middle school students' accuracy at evaluating the quality of their recall. , 2009, Journal of experimental psychology. Applied.

[29]  A. Koriat.  Illusions of Knowing: The Link between Knowledge and Metaknowledge , 1998 .

[30]  Robert A. Bjork,et al.  The promise and perils of self-regulated study , 2007, Psychonomic bulletin & review.

[31]  Jeffrey D. Karpicke,et al.  When and why do retrieval attempts enhance subsequent encoding? , 2012, Memory & cognition.

[32]  C. A. Weaver,et al.  Processing similarity does not improve metamemory: evidence against transfer-appropriate monitoring. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[33]  Asher Koriat,et al.  Predicting one's own forgetting: the role of experience-based and theory-based processes. , 2004, Journal of experimental psychology. General.

[34]  J. Luis Guasch,et al.  Costs and benefits , 2019, Joint Audit 2019 – Enhancing Tax Co-operation and Improving Tax Certainty.

[35]  John Dunlosky,et al.  Does the Sensitivity of Judgments of Learning (JOLs) to the Effects of Various Study Activities Depend on When the JOLs Occur , 1994 .

[36]  John Dunlosky,et al.  Study strategies of college students: Are self-testing and scheduling related to achievement? , 2011, Psychonomic Bulletin & Review.

[37]  Trudy W. Banta,et al.  Promise and perils , 1988 .

[38]  Jeffrey D. Karpicke,et al.  Test-Enhanced Learning , 2006, Psychological science.

[39]  T. O. Nelson,et al.  When People's Judgments of Learning (JOLs) are Extremely Accurate at Predicting Subsequent Recall: The “Delayed-JOL Effect” , 1991 .

[40]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[41]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[42]  A. C. Butler,et al.  The critical role of retrieval practice in long-term retention , 2011, Trends in Cognitive Sciences.

[43]  Norbert M. Seel,et al.  Encyclopedia of the sciences of learning , 2012 .

[44]  Matthew G. Rhodes,et al.  Multiple bases for young and older adults' judgments of learning in multitrial learning. , 2012, Psychology and aging.

[45]  Janet Metcalfe,et al.  Delaying judgments of learning affects memory, not metamemory , 2003, Memory & cognition.

[46]  Predicting memory benefits in the production effect: the use and misuse of self-generated distinctive cues when making judgments of learning , 2013, Memory & cognition.

[47]  Eugene B. Zechmeister,et al.  Judgments of knowing: The influence of retrieval practice. , 1980 .

[48]  Thomas A. Schreiber,et al.  The University of South Florida free association, rhyme, and word fragment norms , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[49]  John Dunlosky,et al.  Toward a general model of self-regulated study: An analysis of selection of items for study and self-paced study time. , 1999 .

[50]  A. Koriat.  Monitoring one's own knowledge during study : A cue-utilization approach to judgments of learning , 1997 .

[51]  Jeffrey D. Karpicke,et al.  Metacognitive control and strategy selection: deciding to practice retrieval during learning. , 2009, Journal of experimental psychology. General.

[52]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[53]  T. O. Nelson,et al.  Why investigate metacognition , 1994 .

[54]  Fredrik U. Jönsson,et al.  The testing effect as a function of explicit testing instructions and judgments of learning. , 2012, Experimental psychology.

[55]  J. Dunlosky,et al.  The sensitivity of judgment-of-learning resolution to past test performance, new learning, and forgetting , 2011, Memory & cognition.

[56]  John Dunlosky,et al.  Improving students’ self-evaluation of learning for key concepts in textbook materials , 2007 .

[57]  Lisa K. Son,et al.  Metacognitive and control strategies in study-time allocation. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[58]  John Dunlosky,et al.  Agenda-based regulation of study-time allocation: when agendas override item-based monitoring. , 2009, Journal of experimental psychology. General.

[59]  T. O. Nelson,et al.  Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect , 1992, Memory & cognition.

[60]  Lisa K. Son,et al.  Learners’ choices and beliefs about self-testing , 2009, Memory.

[61]  Barbara A. Spellman,et al.  Measuring memory and metamemory: Theoretical and statistical problems with assessing learning (in general) and using gamma (in particular) to do so. , 2008 .

[62]  Matthew G. Rhodes,et al.  Metacognitive illusions for auditory information: Effects on monitoring and control , 2009, Psychonomic bulletin & review.

[63]  R. Bjork,et al.  Why Tests Appear to Prevent Forgetting: A Distribution-Based Bifurcation Model. , 2011 .