Investigating the testing effect: Retrieval as a characteristic of effective study strategies

Abstract Presently, the most common approach to examining the testing effect is using a free recall form of retrieval practice. In this experiment, we compared free recall to other retrieval-based study strategies including practice quizzing, test-generation, and keyword. We also examined the possible benefit of coupling these retrieval-based strategies with free recall. A total of 338 undergraduates were randomly assigned to one of the nine conditions: a repeated retrieval (study-test) learning condition paired with one of the four retrieval-based strategies, a single retrieval (study-study) learning condition paired with a retrieval-based strategy, or a rehearsal (study-study-study) condition. Following a 7-day delay, students completed a test assessing retention of information learned. There was a significant interaction between learning condition (repeated vs. single retrieval practice) and type of retrieval-based strategy. Free recall and practice quizzing were the most effective types of retrieval practice, and coupling testgeneration and practice quizzing with free recall led to significant benefits in performance.

[1]  Adam L. Putnam,et al.  Does response mode affect amount recalled or the magnitude of the testing effect? , 2013, Memory & cognition.

[2]  Jeffrey D. Karpicke,et al.  Covert retrieval practice benefits retention as much as overt retrieval practice. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[3]  Ryan C. Leach,et al.  The Generation Effect Revisited: Fewer Generation Constraints Enhances Item and Context Memory , 2017 .

[4]  Mark A. McDaniel,et al.  A contextual account of the generation effect: A three-factor theory ☆ , 1988 .

[5]  John W. Tukey,et al.  Controlling Error in Multiple Comparisons, with Examples from State-to-State Differences in Educational Achievement , 1999 .

[6]  Margaret H. Thomas,et al.  Keyword mnemonic and retention of second-language vocabulary words. , 1992 .

[7]  E. Tulving,et al.  Effects of Temporal and Semantic Encoding in Immediate Recall upon Subsequent Retrieval. , 1974 .

[8]  John Dunlosky,et al.  Influences of metamemory on performance predictions for text , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[9]  Harold Pashler,et al.  Testing beyond words: Using tests to enhance visuospatial map learning , 2007, Psychonomic bulletin & review.

[10]  David J. Therriault,et al.  Accuracy of metacognitive monitoring affects learning of texts. , 2003 .

[11]  Jeffrey D. Karpicke.  Retrieval-Based Learning , 2012 .

[12]  John L Dobson,et al.  Self-testing produces superior recall of both familiar and unfamiliar muscle information. , 2015, Advances in physiology education.

[13]  Jeffrey D. Karpicke,et al.  Retrieval-based learning: The need for guided retrieval in elementary school children , 2014 .

[14]  Annemarie Sullivan Palincsar,et al.  Instructing Comprehension-Fostering Activities in Interactive Learning Situations , 2013 .

[15]  Christine L. Bae,et al.  Using whiteboards to support college students' learning of complex physiological concepts. , 2017, Advances in physiology education.

[16]  Jonathan W. Kelly,et al.  Tests enhance retention and transfer of spatial learning , 2012, Psychonomic bulletin & review.

[17]  P. A. Kolers,et al.  Procedures of mind. , 1984 .

[18]  C. D. Mercer,et al.  Toward Independence with Keyword Mnemonics , 1992 .

[19]  Kathleen B McDermott,et al.  Retrieval-induced facilitation: initially nontested material can benefit from prior testing of related material. , 2006, Journal of experimental psychology. General.

[20]  Keith W. Thiede,et al.  Summarizing can improve metacomprehension accuracy , 2003 .

[21]  M. McDaniel,et al.  Selective Benefits of Question Self-Generation and Answering for Remembering Expository Text , 2012 .

[22]  R. Atkinson.  Mnemotechnics in Second Language Learning. , 1975 .

[23]  John Dunlosky,et al.  Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention , 2012 .

[24]  Shana K. Carpenter,et al.  Semantic information activated during retrieval contributes to later retention: Support for the mediator effectiveness hypothesis of the testing effect. , 2011, Journal of experimental psychology. Learning, memory, and cognition.

[25]  Alexander Renkl,et al.  Enhancing learning by retrieval: Enriching free recall with elaborative prompting , 2017 .

[26]  John R. Anderson,et al.  Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect , 2005, Cogn. Sci..

[27]  G. A. Allen,et al.  Effects of recall tests on long-term retention of paired associates , 1969 .

[28]  Jeffrey D. Karpicke,et al.  Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping , 2011, Science.

[29]  H. Pashler,et al.  Recent Research on Human Learning Challenges Conventional Instructional Strategies , 2010 .

[30]  Harold Pashler,et al.  Journal of Applied Research in Memory and Cognition Is the Benefit of Retrieval Practice Modulated by Motivation? , 2022 .

[31]  David A. Balota,et al.  Expanded vs. Equal Interval Spaced Retrieval Practice: Exploring Different Schedules of Spacing and Retention Interval in Younger and Older Adults , 2008, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[32]  Harold Pashler,et al.  Is temporal spacing of tests helpful even when it inflates error rates? , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[33]  K. McDermott.  Paradoxical effects of testing: Repeated retrieval attempts enhance the likelihood of later accurate and false recall , 2006, Memory & cognition.

[34]  John D. Bransford,et al.  Levels of processing versus transfer appropriate processing , 1977 .

[35]  Christopher A. Rowland,et al.  Benefits of testing for nontested information: Retrieval-induced facilitation of episodically bound material , 2014, Psychonomic bulletin & review.

[36]  Henry L. Roediger,et al.  Expectancy of an open-book test decreases performance on a delayed closed-book test , 2011, Memory.

[37]  M. McDaniel,et al.  Test-enhanced learning in the classroom: long-term improvements from quizzing. , 2011, Journal of experimental psychology. Applied.

[38]  Harold Pashler,et al.  Using tests to enhance 8th grade students' retention of U.S. history facts , 2009 .

[39]  H. Pashler,et al.  Manipulations of Choice Familiarity in Multiple-Choice Testing Support a Retrieval Practice Account of the Testing Effect. , 2014 .

[40]  A. Campos,et al.  Receptive and Productive Recall with the Keyword Mnemonics in Bilingual Students , 2014 .

[41]  Fraser A. Bleasdale,et al.  Retrieval difficulty and subsequent recall , 1973, Memory & cognition.

[42]  Christopher A. Rowland.  The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. , 2014, Psychological bulletin.

[43]  Michael Pressley,et al.  Transfer of a mnemonic keyword strategy at two age levels. , 1980 .

[44]  Juana M. Ortega-Tudela,et al.  Further evidence that concept mapping is not better than repeated retrieval as a tool for learning from texts , 2015 .

[45]  Henry L. Roediger,et al.  Testing improves long-term retention in a simulated classroom setting , 2007 .

[46]  J. Bartlett,et al.  Effects of Immediate Testing on Delayed Retrieval: Search and Recovery Operations with Four Types of Cue. , 1977 .

[47]  John Dunlosky,et al.  Applied Metacognition: Influence of practice tests on the accuracy of predicting memory performance for paired associates, sentences, and text material , 2002 .

[48]  Monica H. Lamm,et al.  A Classroom Study on the Relationship Between Student Achievement and Retrieval-Enhanced Learning , 2016 .

[49]  Mark A. McDaniel,et al.  Classroom-based programs of retrieval practice reduce middle school and high school students' test anxiety , 2014 .

[50]  Katherine A. Rawson,et al.  The Status of the Testing Effect for Complex Materials: Still a Winner , 2015 .

[51]  John Dunlosky,et al.  Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology , 2012 .

[52]  B. Underwood,et al.  Response dominance and rate of learning paired associates. , 1960, The Journal of general psychology.

[53]  M. Pressley,et al.  The Mnemonic Keyword Method , 1982 .

[54]  Jeffrey D. Karpicke,et al.  Test-Enhanced Learning , 2006, Psychological science.

[55]  T. O. Nelson,et al.  Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect , 1992, Memory & cognition.

[56]  M. McDaniel,et al.  Strategic and automatic processes in prospective memory retrieval: a multiprocess framework , 2000 .

[57]  Edward Vul,et al.  What types of learning are enhanced by a cued recall test? , 2006, Psychonomic bulletin & review.

[58]  N. J. Slamecka,et al.  The Generation Effect: Delineation of a Phenomenon , 1978 .

[59]  Ismael Pascual-Nieto,et al.  Computer-assisted assessment of free-text answers , 2009, The Knowledge Engineering Review.

[60]  Leith Sly,et al.  Practice Tests as Formative Assessment Improve Student Performance on Computer‐managed Learning Assessments , 1999 .

[61]  John Dunlosky,et al.  How accurately can students evaluate the quality of self-generated examples of declarative concepts? Not well, and feedback does not help , 2016 .

[62]  Yana Weinstein,et al.  A comparison of study strategies for passages: rereading, answering questions, and generating questions. , 2010, Journal of experimental psychology. Applied.

[63]  B. Rosenshine,et al.  Teaching Students to Generate Questions: A Review of the Intervention Studies , 1996 .

[64]  Katherine A. Rawson,et al.  Why Testing Improves Memory: Mediator Effectiveness Hypothesis , 2010, Science.

[65]  Larry L. Jacoby,et al.  Toward a generate/recognize model of performance on direct and indirect tests of memory ☆ , 1990 .

[66]  G. Bower,et al.  Demonstrations of a generation effect in context memory , 2001, Memory & cognition.

[67]  Henry L. Roediger,et al.  Test format and corrective feedback modify the effect of testing on long-term retention , 2007 .

[68]  Jeffrey D. Karpicke,et al.  Learning with retrieval-based concept mapping. , 2014 .

[69]  Jeffrey D. Karpicke,et al.  The effect of type and timing of feedback on learning from multiple-choice tests. , 2007, Journal of experimental psychology. Applied.

[70]  Jeffrey D. Karpicke,et al.  Guided Retrieval Practice of Educational Materials Using Automated Scoring , 2014 .

[71]  John Dunlosky,et al.  Optimizing schedules of retrieval practice for durable and efficient learning: how much is enough? , 2011, Journal of experimental psychology. General.

[72]  Mark A. McDaniel,et al.  Quizzing in Middle-School Science: Successful Transfer Performance on Classroom Exams , 2013 .

[73]  Elizabeth J Marsh,et al.  Memorial consequences of multiple-choice testing on immediate and delayed tests , 2010, Memory & cognition.

[74]  Katherine A. Rawson,et al.  Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory? , 2009 .

[75]  A. C. Butler,et al.  The critical role of retrieval practice in long-term retention , 2011, Trends in Cognitive Sciences.

[76]  Elizabeth J Marsh,et al.  The positive and negative consequences of multiple-choice testing. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[77]  Genna Angello,et al.  Multiple-Choice Tests Exonerated, at Least of Some Charges , 2012, Psychological science.

[78]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[79]  Patrice M. Bain,et al.  The Value of Applied Research: Retrieval Practice Improves Classroom Learning and Recommendations from a Teacher, a Principal, and a Scientist , 2012 .

[80]  Jeffrey D. Karpicke,et al.  Retrieval practice with short-answer, multiple-choice, and hybrid tests , 2014, Memory.

[81]  L. E. Bourne,et al.  Remembering the levels of information in words , 1978 .

[82]  Kirsten Berthold,et al.  Effects of incorporating retrieval into learning tasks: The complexity of the tasks matters , 2017 .

[83]  John Sweller,et al.  Not New, but Nearly Forgotten: the Testing Effect Decreases or even Disappears as the Complexity of Learning Materials Increases , 2015 .

[84]  Mark A. McDaniel,et al.  Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and Placement. , 2011 .

[85]  John Dunlosky,et al.  The Power of Successive Relearning: Improving Performance on Course Exams and Long-Term Retention , 2013 .

[86]  Jeffrey D. Karpicke,et al.  The Power of Testing Memory Basic Research and Implications for Educational Practice , 2006 .

[87]  Bruce Rogers.  TOEFL CBT Success , 2000 .

[88]  John L. Dobson,et al.  Self-testing promotes superior retention of anatomy and physiology information , 2015, Advances in health sciences education : theory and practice.

[89]  Olusola O. Adesope,et al.  Rethinking the Use of Tests: A Meta-Analysis of Practice Testing , 2017 .

[90]  Jeffrey D. Karpicke,et al.  Separate mnemonic effects of retrieval practice and elaborative encoding , 2012 .

[91]  J. H. Neely,et al.  Testing enhances both encoding and retrieval for both tested and untested items , 2017, Quarterly journal of experimental psychology.

[92]  Jeffrey D. Karpicke,et al.  Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? , 2009, Memory.

[93]  Mark A. McDaniel,et al.  The testing effect with authentic educational materials: A cautionary note , 2014 .

[94]  Mirella Wyra,et al.  The mnemonic keyword method: The effects of bidirectional retrieval training and of ability to image on foreign language vocabulary recall , 2007 .

[95]  Henry L. Roediger,et al.  Repeated retrieval during learning is the key to long-term retention , 2007 .

[96]  Shana K. Carpenter,et al.  Enhancing learning and retarding forgetting: Choices and consequences , 2007, Psychonomic bulletin & review.

[97]  Andrew C Butler,et al.  Repeated testing produces superior transfer of learning relative to repeated studying. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[98]  Katherine A. Rawson,et al.  Relearning attenuates the benefits and costs of spacing. , 2013, Journal of experimental psychology. General.

[99]  Jeffrey D. Karpicke,et al.  The Testing Effect Is Alive and Well with Complex Materials , 2015 .

[100]  Cheryl I. Johnson,et al.  A Testing Effect with Multimedia Learning , 2009 .