Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning?

The development of students’ higher order learning is a critical component of education. For decades, educators and scientists have engaged in an ongoing debate about whether higher order learning can only be enhanced by building a base of factual knowledge (analogous to Bloom’s taxonomy) or whether higher order learning can be enhanced directly by engaging in complex questioning and materials. The relationship between fact learning and higher order learning is often speculated, but empirically unknown. In this study, middle school students and college students engaged in retrieval practice with fact questions, higher order questions, or a mix of question types to examine the optimal type of retrieval practice for enhancing higher order learning. In laboratory and K-12 settings, retrieval practice consistently increased delayed test performance, compared with rereading or no quizzes. Critically, higher order and mixed quizzes improved higher order test performance, but fact quizzes did not. Contrary to popular intuition about higher order learning and Bloom’s taxonomy, building a foundation of knowledge via fact-based retrieval practice may be less potent than engaging in higher order retrieval practice, a key finding for future research and classroom application.

[1]  Frank J. Bonello,et al.  Taking Sides: Clashing Views on Controversial Economic Issues , 1988 .

[2]  Keith B. Lyle,et al.  Retrieving Essential Material at the End of Lectures Improves Performance on Statistics Exams , 2011 .

[3]  C. Skinner,et al.  Interspersing Additional Brief, Easy Problems to Increase Assignment Preference on Mathematics Reading Problems , 1998 .

[4]  John Sweller,et al.  Not New, but Nearly Forgotten: the Testing Effect Decreases or even Disappears as the Complexity of Learning Materials Increases , 2015 .

[5]  Mark A. McDaniel,et al.  Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and Placement. , 2011 .

[6]  Andrew C. Butler,et al.  Retrieving and Applying Knowledge to Different Examples Promotes Transfer of Learning , 2017, Journal of experimental psychology. Applied.

[7]  R. Bjork,et al.  Learning Concepts and Categories , 2008, Psychological science.

[8]  Christopher N. Wahlheim,et al.  Test-enhanced learning of natural concepts: effects on recognition memory, classification, and metacognition. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[9]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[10]  Shana K. Carpenter,et al.  Cue strength as a moderator of the testing effect: the benefits of elaborative retrieval. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[11]  Jonas Gloeckner,et al.  Education And Learning To Think , 2016 .

[12]  Tyler A. Kummer,et al.  Teaching to the Test…or Testing to Teach: Exams Requiring Higher Order Thinking Skills Encourage Greater Conceptual Understanding , 2014 .

[13]  H. Pashler,et al.  Recent Research on Human Learning Challenges Conventional Instructional Strategies , 2010 .

[14]  Christopher A. Rowland The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. , 2014, Psychological bulletin.

[15]  C. Skinner,et al.  Improving assignment preference through interspersing additional problems: Brief versus easy problems , 1996 .

[16]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[17]  Mark J. Gierl,et al.  Developing, Analyzing, and Using Distractors for Multiple-Choice Tests in Education: A Comprehensive Review , 2017 .

[18]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[19]  F. Bartlett Thinking: An Experimental and Social Study , 1958 .

[20]  Katherine A. Rawson,et al.  The Status of the Testing Effect for Complex Materials: Still a Winner , 2015 .

[21]  Henry L. Roediger,et al.  Test format and corrective feedback modify the effect of testing on long-term retention , 2007 .

[22]  E. Hirsch,et al.  The Schools We Need: And Why We Don't Have Them , 1996 .

[23]  J. Sweller Cognitive Load Theory: Cognitive Load Theory: Recent Theoretical Advances , 2010 .

[24]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[25]  Diane Hart,et al.  Authentic Assessment: A Handbook for Educators , 1992 .

[26]  Shana K. Carpenter Testing Enhances the Transfer of Learning , 2012 .

[27]  Doug Rohrer,et al.  The effects of interleaved practice , 2010 .

[28]  Douglas P Larsen,et al.  The importance of seeing the patient: test-enhanced learning with standardized patients and written tests improves clinical application of knowledge , 2013, Advances in health sciences education : theory and practice.

[29]  E. Yalow Educational psychology: A cognitive view. 2nd ed. , 1978 .

[30]  M. McDaniel,et al.  Make It Stick , 2014 .

[31]  Jason R. Finley,et al.  Benefits from retrieval practice are greater for students with lower working memory capacity , 2017, Memory.

[32]  S. Geisser,et al.  On methods in the analysis of profile data , 1959 .

[33]  John Dunlosky,et al.  Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology , 2012 .

[34]  Herman Aguinis,et al.  Cautionary Note on Reporting Eta-Squared Values from Multifactor ANOVA Designs , 2004 .

[35]  Henry L. Roediger,et al.  Expectancy of an open-book test decreases performance on a delayed closed-book test , 2011, Memory.

[36]  Timothy C. Rickard,et al.  Transfer of Test-Enhanced Learning: Meta-Analytic Review and Synthesis , 2018, Psychological bulletin.

[37]  K. Holyoak,et al.  Analogical problem solving , 1980, Cognitive Psychology.

[38]  Roland Brünken,et al.  Cognitive Load Theory: THEORY , 2010 .

[39]  Larry Cuban,et al.  Policy and Research Dilemmas in the Teaching of Reasoning: Unplanned Designs , 1984 .

[40]  L. E. Bourne,et al.  Remembering the levels of information in words , 1978 .

[41]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[42]  W. James Scientific Books: Talks to Teachers on Psychology, and to Students on Some of Life's Ideals , 2013 .

[43]  L. Resnick Education and Learning to Think , 1987 .

[44]  Michael E. Martinez Cognition and the question of test item format , 1999 .

[45]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[46]  Elizabeth J. Marsh,et al.  Chapter 02: Learning from the Test: Dos and Don'ts for Using Multiple-Choice Tests , 2014 .

[47]  Katherine A. Rawson,et al.  Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory? , 2009 .

[48]  H. Pashler,et al.  The influence of retrieval on retention , 1992, Memory & cognition.

[49]  J. Dewey Democracy and education : an introduction to the philosophy of education , 1961 .

[50]  Jeffrey D. Karpicke,et al.  Test-Enhanced Learning , 2006, Psychological science.

[51]  Doug Rohrer,et al.  Tests enhance the transfer of learning. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[52]  Url PubMed Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping , 2012 .

[53]  Patrice M. Bain,et al.  The Value of Applied Research: Retrieval Practice Improves Classroom Learning and Recommendations from a Teacher, a Principal, and a Scientist , 2012 .

[54]  J. P. Slavinsky,et al.  Integrating Cognitive Science and Technology Improves Learning in a STEM Classroom , 2014 .

[55]  Andrew C Butler,et al.  Repeated testing produces superior transfer of learning relative to repeated studying. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[56]  D. Ausubel,et al.  IN DEFENSE OF VERBAL LEARNING , 1961 .

[57]  John D. Bransford,et al.  Levels of processing versus transfer appropriate processing , 1977 .

[58]  Kathleen B McDermott,et al.  Retrieval-induced facilitation: initially nontested material can benefit from prior testing of related material. , 2006, Journal of experimental psychology. General.

[59]  Jennifer Wiley,et al.  Testing the limits of testing effects using completion tests , 2011, Memory.

[60]  Mark A. McDaniel,et al.  Classroom-based programs of retrieval practice reduce middle school and high school students' test anxiety , 2014 .

[61]  Alfie Kohn,et al.  The Schools Our Children Deserve: Moving Beyond Traditional Classrooms and "Tougher Standards" , 1999 .

[62]  Fraser A. Bleasdale,et al.  Retrieval difficulty and subsequent recall , 1973, Memory & cognition.

[63]  R. Bjork Memory and metamemory considerations in the training of human beings. , 1994 .

[64]  Henry L. Roediger,et al.  Generalizing test-enhanced learning from the laboratory to the classroom , 2007, Psychonomic bulletin & review.

[65]  Catherine E. Overson,et al.  Applying science of learning in education: Infusing psychological science into the curriculum , 2014 .

[66]  Kelli M Taylor,et al.  The effects of overlearning and distributed practise on the retention of mathematics knowledge , 2006 .

[67]  John D. Bransford,et al.  Teaching thinking and problem solving: Research foundations. , 1986 .

[68]  Charles Degarmo Talks to Teachers on Psychology, and to Students on some of Life's Ideals , 1899 .

[69]  Henry L. Roediger,et al.  Repeated retrieval during learning is the key to long-term retention , 2007 .

[70]  M. McDaniel,et al.  Test-enhanced learning in the classroom: long-term improvements from quizzing. , 2011, Journal of experimental psychology. Applied.

[71]  M. Ewers,et al.  Different rates of forgetting following study versus test trials , 2003, Memory.

[72]  Jeffrey D. Karpicke,et al.  The Power of Testing Memory Basic Research and Implications for Educational Practice , 2006 .

[73]  W. Rice ANALYZING TABLES OF STATISTICAL TESTS , 1989, Evolution; international journal of organic evolution.

[74]  Douglas P Larsen,et al.  Test-enhanced learning , 2013 .

[75]  Charlotte Ringsted,et al.  The effect of testing on skills learning , 2009, Medical education.

[76]  Mark A McDaniel,et al.  Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. , 2014, Journal of experimental psychology. Applied.

[77]  A. M. White The Process of Education , 1994 .

[78]  Mark A. McDaniel,et al.  The Limited Benefits of Rereading Educational Texts. , 2009 .

[79]  Daniel T. Willingham,et al.  Why Don't Students Like School?: A Cognitive Scientist Answers Questions About How the Mind Works and What It Means for the Classroom , 2009 .

[80]  Bruce Alberts From the National Academies. , 2002, Cell biology education.

[81]  F. Paas Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. , 1992 .

[82]  Olusola O. Adesope,et al.  Rethinking the Use of Tests: A Meta-Analysis of Practice Testing , 2017 .

[83]  Henry L. Roediger,et al.  Examining the Testing Effect with Open-and Closed-book Tests , 2022 .

[84]  Genna Angello,et al.  Multiple-Choice Tests Exonerated, at Least of Some Charges , 2012, Psychological science.

[85]  J. K. Chan,et al.  When does Retrieval Induce Forgetting and When does it Induce Facilitation? Implications for Retrieval Inhibition, Testing Effect, and Text Processing , 2009 .

[86]  Susan M. Barnett,et al.  When and where do we apply what we learn? A taxonomy for far transfer. , 2002, Psychological bulletin.

[87]  E. Bjork,et al.  Optimizing multiple-choice tests as tools for learning , 2014, Memory & Cognition.

[88]  Mark A. McDaniel,et al.  Quizzing in Middle-School Science: Successful Transfer Performance on Classroom Exams , 2013 .

[89]  Elizabeth Ligon Bjork,et al.  Multiple-choice testing as a desirable difficulty in the classroom , 2014 .

[90]  Henry L. Roediger,et al.  Benefits of testing memory: Best practices and boundary conditions. , 2010 .

[91]  Jeffrey D. Karpicke,et al.  The Testing Effect Is Alive and Well with Complex Materials , 2015 .

[92]  Katherine A. Rawson,et al.  Why Testing Improves Memory: Mediator Effectiveness Hypothesis , 2010, Science.

[93]  Arthur C. Graesser,et al.  Organizing Instruction and Study to Improve Student Learning. IES Practice Guide. NCER 2007-2004. , 2007 .

[94]  Denis Cousineau,et al.  Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson's method , 2005 .