The influence of delaying judgments of learning on metacognitive accuracy: a meta-analytic review.

Many studies have examined the accuracy of predictions of future memory performance solicited through judgments of learning (JOLs). Among the most robust findings in this literature is that delaying predictions serves to substantially increase the relative accuracy of JOLs compared with soliciting JOLs immediately after study, a finding termed the delayed JOL effect. The meta-analyses reported in the current study examined the predominant theoretical accounts as well as potential moderators of the delayed JOL effect. The first meta-analysis examined the relative accuracy of delayed compared with immediate JOLs across 4,554 participants (112 effect sizes) through gamma correlations between JOLs and memory accuracy. Those data showed that delaying JOLs leads to robust benefits to relative accuracy (g = 0.93). The second meta-analysis examined memory performance for delayed compared with immediate JOLs across 3,807 participants (98 effect sizes). Those data showed that delayed JOLs result in a modest but reliable benefit for memory performance relative to immediate JOLs (g = 0.08). Findings from these meta-analyses are well accommodated by theories suggesting that delayed JOL accuracy reflects access to more diagnostic information from long-term memory rather than being a by-product of a retrieval opportunity. However, these data also suggest that theories proposing that the delayed JOL effect results from a memorial benefit or the match between the cues available for JOLs and those available at test may also provide viable explanatory mechanisms necessary for a comprehensive account.

[1]  M. Kennedy Self-monitoring recall during two tasks after traumatic brain injury: a preliminary study. , 2004, American journal of speech-language pathology.

[2]  John Dunlosky,et al.  Does the Sensitivity of Judgments of Learning (JOLs) to the Effects of Various Study Activities Depend on When the JOLs Occur , 1994 .

[3]  T. O. Nelson,et al.  A comparison of current measures of the accuracy of feeling-of-knowing predictions. , 1984, Psychological bulletin.

[4]  Laura E. Knouse,et al.  Does ADHD in Adults Affect the Relative Accuracy of Metamemory Judgments? , 2006, Journal of attention disorders.

[5]  John Dunlosky,et al.  Delaying students' metacognitive monitoring improves their accuracy in predicting their recognition performance. , 1994 .

[6]  T. O. Nelson,et al.  Measuring ordinal association in situations that contain tied scores. , 1996, Psychological bulletin.

[7]  Janet Metcalfe,et al.  The role of memory for past test in the underconfidence with practice effect. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Jennifer Wiley,et al.  Foundational Studies 1-1-2005 Understanding the Delayed-Keyword Effect on Metacomprehension Accuracy , 2013 .

[9]  Asher Koriat,et al.  The effects of encoding fluency and retrieval fluency on judgments of learning , 2005 .

[10]  T. O. Nelson,et al.  Anchoring Effects in the Absolute Accuracy of Immediate versus Delayed Judgments of Learning. , 2004 .

[11]  Barbara A. Spellman,et al.  Measuring memory and metamemory: Theoretical and statistical problems with assessing learning (in general) and using gamma (in particular) to do so. , 2008 .

[12]  Lola L. Cuddy,et al.  Discrimination of item strength at time of presentation , 1969 .

[13]  T. O. Nelson,et al.  When People's Judgments of Learning (JOLs) are Extremely Accurate at Predicting Subsequent Recall: The “Delayed-JOL Effect” , 1991 .

[14]  L. Hedges Distribution Theory for Glass's Estimator of Effect size and Related Estimators , 1981 .

[15]  P. Scheck HOW DO JUDGMENTS OF INTENTION (JOIS) DIFFER FROM JUDGMENTS OF LEARNING (JOLS) , 2006 .

[16]  Similarity between the Cue for Judgments of Learning (JOL) and the Cue for Test Is Not the Primary Determinant of JOL Accuracy , 1997 .

[17]  A. Miyake,et al.  Models of Working Memory: Mechanisms of Active Maintenance and Executive Control , 1999 .

[18]  Barbara A. Spellman,et al.  When Predictions Create Reality: Judgments of Learning May Alter What They Are Intended to Assess , 1992 .

[19]  T. O. Nelson,et al.  Developmental trends in children's memory monitoring , 2000 .

[20]  C. A. Weaver,et al.  Individual differences in metacognition: Evidence against a general metacognitive ability , 2000, Memory & cognition.

[21]  T. O. Nelson,et al.  Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect , 1992, Memory & cognition.

[22]  J. Dunlosky,et al.  The effects of categorical relatedness on judgements of learning (JOLs) , 2006, Memory.

[23]  A. Koriat Monitoring one's own knowledge during study : A cue-utilization approach to judgments of learning , 1997 .

[24]  Lisa K. Son,et al.  Judgments of learning: Evidence for a two-stage process , 2005, Memory & cognition.

[25]  F. Craik,et al.  The Oxford handbook of memory , 2006 .

[26]  Matthew G. Rhodes,et al.  Metacognitive illusions for auditory information: Effects on monitoring and control , 2009, Psychonomic bulletin & review.

[27]  H. Pashler,et al.  Distributed practice in verbal recall tasks: A review and quantitative synthesis. , 2006, Psychological bulletin.

[28]  Klaus Oberauer,et al.  Control of the contents of working memory--a comparison of two paradigms and two age groups. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[29]  R. Griffiths,et al.  Drugs, memory, and metamemory: a dose-effect study with lorazepam and scopolamine. , 2005, Experimental and clinical psychopharmacology.

[30]  Matthew G. Rhodes,et al.  Memory predictions are influenced by perceptual information: evidence for metacognitive illusions. , 2008, Journal of experimental psychology. General.

[31]  M. Borenstein Effect sizes for continuous data. , 2009 .

[32]  William R. Shadish,et al.  Combining estimates of effect size. , 1994 .

[33]  Michael J. Serra,et al.  Does retrieval fluency contribute to the underconfidence-with-practice effect? , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[34]  C. A. Weaver,et al.  Judgments of Learning at Delays: Shifts in Response Patterns or Increased Metamemory Accuracy? , 1997 .

[35]  J. Metcalfe,et al.  Familiarity and retrieval processes in delayed judgments of learning. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[36]  A. Koriat,et al.  Metacognitive Judgments and their Accuracy , 2002 .

[37]  J. Metcalfe,et al.  A Region of Proximal Learning Model of Study Time Allocation Journal of Memory and Language , 2005 .

[38]  Christopher N. Wahlheim,et al.  Predicting memory performance under conditions of proactive interference: Immediate and delayed judgments of learning , 2011, Memory & cognition.

[39]  Asher Koriat,et al.  The intricate relationships between monitoring and control in metacognition: lessons for the cause-and-effect relation between subjective experience and behavior. , 2006, Journal of experimental psychology. General.

[40]  T. O. Nelson,et al.  Tradeoff of semantic relatedness and degree of overlearning: differential effects on metamemory and on long-term retention. , 1997, Acta psychologica.

[41]  T. O. Nelson,et al.  Allocation of self-paced study time and the "labor-in-vain effect". , 1988, Journal of experimental psychology. Learning, memory, and cognition.

[42]  J. Dunlosky,et al.  Age-related differences in absolute but not relative metamemory accuracy. , 1997, Psychology and aging.

[43]  K. Yorkston,et al.  The effects of frontal injury on “on-line” self-monitoring during verbal learning by adults with diffuse brain injury , 2004 .

[44]  M. Kennedy,et al.  Predictions of recall and study strategy decisions after diffuse brain injury , 2003, Brain injury.

[45]  J. Metcalfe,et al.  Metacognition : knowing about knowing , 1994 .

[46]  Ayanna K. Thomas,et al.  Metacomprehension for educationally relevant materials: Dramatic effects of encoding-retrieval interactions , 2007, Psychonomic bulletin & review.

[47]  John Dunlosky,et al.  Applied Metacognition: Influence of practice tests on the accuracy of predicting memory performance for paired associates, sentences, and text material , 2002 .

[48]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[49]  Pierre-Jean Marescaux,et al.  Metacognition : process, function and use , 2002 .

[50]  C. A. Weaver,et al.  Enhanced metamemory at delays: why do judgments of learning improve over time? , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[51]  R. Bjork,et al.  Illusions of competence in monitoring one's knowledge during study. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[52]  W. Kelemen,et al.  Metamemory Cues and Monitoring Accuracy: Judging What You Know and What You Will Know , 2000 .

[53]  T. O. Nelson,et al.  Multiple study trials and judgments of learning. , 2003, Acta psychologica.

[54]  Caren M Rotello,et al.  Sources of bias in the Goodman-Kruskal gamma coefficient measure of association: implications for studies of metacognitive processes. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[55]  Michael J. Serra,et al.  Do Older Adults Show Less Confidence in Their Monitoring of Learning? , 2008, Experimental aging research.

[56]  Ian Begg,et al.  Memory predictions are based on ease of processing , 1989 .

[57]  Keith W Thiede,et al.  Why do delayed summaries improve metacomprehension accuracy? , 2008, Acta psychologica.

[58]  Katherine A. Rawson,et al.  Second-Order Judgments About Judgments of Learning , 2005 .

[59]  P. Rast,et al.  Age Differences in the Underconfidence-With-Practice Effect , 2009, Experimental aging research.

[60]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[61]  R. Rosenthal The file drawer problem and tolerance for null results , 1979 .

[62]  T. O. Nelson Consciousness and metacognition. , 1996 .

[63]  N. Cowan An embedded-processes model of working memory , 1999 .

[64]  J. Metcalfe,et al.  Evidence that judgments of learning are causally related to study choice , 2008, Psychonomic bulletin & review.

[65]  Matthew G. Rhodes,et al.  Monitoring memory errors: The influence of the veracity of retrieved information on the accuracy of judgements of learning , 2011, Memory.

[66]  T. O. Nelson,et al.  Delayed judgments of learning cause both a decrease in absolute accuracy (calibration) and an increase in relative accuracy (resolution) , 2006, Memory & cognition.

[67]  T. O. Nelson Metamemory: A Theoretical Framework and New Findings , 1990 .

[68]  John Dunlosky,et al.  A revised methodology for research on metamemory: Pre-judgment Recall and Monitoring (PRAM). , 2004, Psychological methods.

[69]  Asher Koriat,et al.  Metacognition and Consciousness 1 Metacognition and Consciousness , 2006 .

[70]  C. A. Weaver,et al.  Processing similarity does not improve metamemory: evidence against transfer-appropriate monitoring. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[71]  Janet Metcalfe,et al.  Delaying judgments of learning affects memory, not metamemory , 2003, Memory & cognition.

[72]  E. Tulving,et al.  Associative encoding and retrieval: Weak and strong cues. , 1970 .

[73]  T. O. Nelson,et al.  Memory Monitoring and the Delayed JOL Effect , 2008 .

[74]  John Dunlosky,et al.  Handbook of Metamemory and Memory , 2013 .

[75]  T. O. Nelson,et al.  How many dimensions underlie judgments of learning and recall? Evidence from state-trace methodology. , 2005, Journal of experimental psychology. General.

[76]  Michael Diaz,et al.  Measurement of relative metamnemonic accuracy. , 2008 .

[77]  Bridgid Finn,et al.  Framing effects on metacognitive monitoring and control , 2008, Memory & cognition.

[78]  Fredrik U. Jönsson,et al.  A model for stochastic drift in memory strength to account for judgments of learning. , 2005, Psychological review.

[79]  J. Metcalfe Metamemory: Theory and data. , 2000 .

[80]  Larry V. Hedges,et al.  Fixed-Effects Models , 2022, The SAGE Encyclopedia of Research Design.

[81]  Asher Koriat,et al.  Illusions of competence during study can be remedied by manipulations that enhance learners’ sensitivity to retrieval conditions at test , 2006, Memory & cognition.

[82]  E. Thompson,et al.  The Cambridge Handbook of Consciousness , 2007 .

[83]  H. Weisberg Sources of Bias , 2010 .

[84]  Claudia M. Roebers,et al.  Children's metamemorial judgments in an event recall task. , 2007, Journal of experimental child psychology.

[85]  T. O. Nelson,et al.  Why investigate metacognition , 1994 .

[86]  C. Hertzog,et al.  Immediate judgments of learning are insensitive to implicit interference effects at retrieval , 2011, Memory & Cognition.

[87]  W. Dunlap,et al.  Meta-Analysis of Experiments With Matched Groups or Repeated Measures Designs , 1996 .

[88]  David J. Therriault,et al.  Accuracy of metacognitive monitoring affects learning of texts. , 2003 .

[89]  Matthew G. Rhodes,et al.  Does the amount of material to be remembered influence judgements of learning (JOLs)? , 2010, Memory.

[90]  Yoonhee Jang THE EFFECT OF PRACTICE WITH TEST ON THE RELATIVE ACCURACY OF JUDGMENTS OF LEARNING , 2006 .

[91]  K. Oberauer Access to information in working memory: exploring the focus of attention. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[92]  Asher Koriat,et al.  Exploring a mnemonic debiasing account of the underconfidence-with-practice effect. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[93]  T. O. Nelson,et al.  Inhalation of 30% nitrous oxide impairs people's learning without impairing people's judgments of what will be remembered. , 1998, Experimental and clinical psychopharmacology.