Evaluating the Theoretic Adequacy and Applied Potential of Computational Models of the Spacing Effect.

The spacing effect is among the most widely replicated empirical phenomena in the learning sciences, and its relevance to education and training is readily apparent. Yet successful applications of spacing effect research to education and training is rare. Computational modeling can provide the crucial link between a century of accumulated experimental data on the spacing effect and the emerging interest in using that research to enable adaptive instruction. In this paper, we review relevant literature and identify 10 criteria for rigorously evaluating computational models of the spacing effect. Five relate to evaluating the theoretic adequacy of a model, and five relate to evaluating its application potential. We use these criteria to evaluate a novel computational model of the spacing effect called the Predictive Performance Equation (PPE). Predictive Performance Equation combines elements of earlier models of learning and memory including the General Performance Equation, Adaptive Control of Thought-Rational, and the New Theory of Disuse, giving rise to a novel computational account of the spacing effect that performs favorably across the complete sets of theoretic and applied criteria. We implemented two other previously published computational models of the spacing effect and compare them to PPE using the theoretic and applied criteria as guides.

[1]  James Niehaus,et al.  Prevention of surgical skill decay. , 2013, Military medicine.

[2]  Robert A. Bjork,et al.  A new theory of disuse and an old theory of stimulus fluctuation , 1992 .

[3]  Michael J Kahana,et al.  Contextual Variability in Free Recall. , 2011, Journal of memory and language.

[4]  H. P. Bahrick,et al.  Retention of Spanish vocabulary over 8 years. , 1987 .

[5]  Gigin Sappena Ginting IMPROVING THE STUDENTS , 2013 .

[6]  Glenn Gunzelmann,et al.  Personalizing Training to Acquire and Sustain Competence Through Use of a Cognitive Model , 2017, HCI.

[7]  Joseph L. Young,et al.  Reinforcement-test intervals in paired-associate learning , 1971 .

[8]  James E. Driskell,et al.  Effect of overlearning on retention. , 1992 .

[9]  Frank E. Ritter,et al.  Learning and Retention , 2013 .

[10]  Benton J. Underwood,et al.  A breakdown of the total-time law in free-recall learning , 1970 .

[11]  Steve Bird,et al.  Effects of distributed practice on the acquisition of second language English syntax , 2010, Applied Psycholinguistics.

[12]  F. N. Dempster,et al.  Spacing effects and their implications for theory and practice , 1989 .

[13]  Michael D. Lee,et al.  A Survey of Model Evaluation Approaches With a Tutorial on Hierarchical Bayesian Methods , 2008, Cogn. Sci..

[14]  Julie L. Booth,et al.  Instructional Complexity and the Science to Constrain It , 2013, Science.

[15]  P. Ackerman Individual differences in skill learning: An integration of psychometric and information processing perspectives. , 1987 .

[16]  Hedderik van Rijn,et al.  An Individual's Rate of Forgetting Is Stable Over Time but Differs Across Materials , 2016, Top. Cogn. Sci..

[17]  R. Schmidt,et al.  New Conceptualizations of Practice: Common Principles in Three Paradigms Suggest New Concepts for Training , 1992 .

[18]  H. P. Bahrick Maintenance of knowledge: Questions about memory we forgot to ask. , 1979 .

[19]  John R. Anderson,et al.  Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect , 2005, Cogn. Sci..

[20]  Marsha C. Lovett,et al.  The Cognitive Science Approach to Learning and Memory , 2017 .

[21]  Peter Grünwald,et al.  Accumulative prediction error and the selection of time series models , 2006 .

[22]  Katherine A. Rawson,et al.  Does the benefit of testing depend on lag, and if so, why? Evaluating the elaborative retrieval hypothesis , 2014, Memory & Cognition.

[23]  H. Pashler,et al.  Distributed practice in verbal recall tasks: A review and quantitative synthesis. , 2006, Psychological bulletin.

[24]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[25]  Kalif E. Vaughn,et al.  Effects of successive relearning on recall: Does relearning override the effects of initial learning criterion? , 2016, Memory & cognition.

[26]  Chris Janiszewski,et al.  A Meta-analysis of the Spacing Effect in Verbal Learning: Implications for Research on Advertising Repetition and Consumer Memory , 2003 .

[27]  Winston Bennett,et al.  The Effect of Distributed Practice on Immediate Posttraining, and Long-Term Performance on a Complex Command-and-Control Simulation Task , 2010 .

[28]  Michael C. Mozer,et al.  Maximizing Students' Retention Via Spaced Review: Practical Guidance From Computational Models Of Memory , 2013, CogSci.

[29]  Timothy D. Lee,et al.  Distribution of Practice in Motor Skill Acquisition: Learning and Performance Effects Reconsidered , 1988 .

[30]  I. Begg,et al.  Repetition and trace interaction: Superadditivity , 1988, Memory & cognition.

[31]  M. Oermann,et al.  Effects of practice on competency in single-rescuer cardiopulmonary resuscitation. , 2014, Medsurg nursing : official journal of the Academy of Medical-Surgical Nurses.

[32]  M. Lee,et al.  Modeling individual differences in cognition , 2005, Psychonomic bulletin & review.

[33]  Kevin A. Gluck,et al.  Understanding and Predicting the Cognitive Effects of Sleep Loss Through Simulation , 2015 .

[34]  Edward L. Thorndike,et al.  The curve of work. , 1912 .

[35]  A. Glenberg,et al.  Spacing repetitions over 1 week , 1980, Memory & cognition.

[36]  Michael C. Mozer,et al.  Predicting and Improving Memory Retention: Psychological Theory Matters in the Big Data Era , 2016 .

[37]  T. Toppino,et al.  The spacing effect in the free recall of homogeneous lists: Present and accounted for , 2002, Memory & cognition.

[38]  David A. Balota,et al.  Age-related differences in the impact of spacing, lag, and retention interval. , 1989 .

[39]  L. R. Peterson,et al.  Effect of spacing presentations on retention of a paired associate over short intervals. , 1963, Journal of experimental psychology.

[40]  H. P. Bahrick,et al.  Maintenance of Foreign Language Vocabulary and the Spacing Effect , 1993 .

[42]  Peter P. J. L. Verkoeijen,et al.  Spacing and Testing Effects: A Deeply Critical, Lengthy, and At Times Discursive Review of the Literature , 2010 .

[43]  Brian H. Ross,et al.  Memory for at least one of two items: Test and failure of several theories of spacing effects , 1978 .

[44]  Katherine A. Rawson,et al.  Relearning attenuates the benefits and costs of spacing. , 2013, Journal of experimental psychology. General.

[45]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[46]  Ed Vul,et al.  Predicting the Optimal Spacing of Study: A Multiscale Context Model of Memory , 2009, NIPS.

[47]  B. Underwood,et al.  Further evidence on the MP-DP effect in free-recall learning , 1972 .

[48]  Frank J. Lee,et al.  Does Learning a Complex Task Have to Be Complex?: A Study in Learning Decomposition , 2001, Cognitive Psychology.

[49]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[50]  E. Erdfelder,et al.  Encoding, maintenance, and retrieval processes in the lag effect: A multinomial processing tree analysis , 2012, Memory.

[51]  John R. Anderson Spanning seven orders of magnitude: a challenge for cognitive modeling , 2002 .

[52]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[53]  Robert L. Greene,et al.  Spacing effects in memory: evidence for a two-process account , 1989 .

[54]  Scott D. Brown,et al.  The power law repealed: The case for an exponential law of practice , 2000, Psychonomic bulletin & review.

[55]  John Dunlosky,et al.  Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology , 2012 .

[56]  Jaehyon Paik,et al.  Evaluating a range of learning schedules: hybrid training schedules may be as good as or better than distributed practice for some tasks , 2016, Ergonomics.

[57]  P. Kellman,et al.  A comparison of adaptive and fixed schedules of practice. , 2016, Journal of experimental psychology. General.

[58]  Wayne D. Gray Cognitive Modeling for Cognitive Engineering , 2008 .

[59]  E. Spruit,et al.  Increasing efficiency of surgical training: effects of spacing practice on skill acquisition and retention in laparoscopy training , 2015, Surgical Endoscopy.

[60]  Winston Bennett,et al.  Factors That Influence Skill Decay and Retention: A Quantitative Review and Analysis , 1998 .

[61]  Slava Kalyuga,et al.  Incorporating Learner Experience into the Design of Multimedia Instruction. , 2000 .

[62]  Robert V. Lindsey,et al.  Improving Students’ Long-Term Knowledge Retention Through Personalized Review , 2014, Psychological science.

[63]  D. Balota,et al.  Age-related differences in the impact of spacing, lag, and retention interval. , 1989, Psychology and aging.

[64]  Dianne C. Berry,et al.  Metacognitive Experience and Transfer of Logical Reasoning , 1983 .

[65]  J. Wixted,et al.  Genuine power curves in forgetting: A quantitative analysis of individual subject forgetting functions , 1997, Memory & cognition.

[66]  John R. Anderson,et al.  Navigating complex decision spaces: Problems and paradigms in sequential choice. , 2014, Psychological bulletin.

[67]  Adam Dubrowski,et al.  Teaching Surgical Skills: What Kind of Practice Makes Perfect?: A Randomized, Controlled Trial , 2006, Annals of surgery.

[68]  W. Estes Statistical theory of distributional phenomena in learning. , 1955, Psychological review.

[69]  B. Barnes,et al.  Creating the practice‐learning environment: using information technology to support a new model of continuing medical education , 1998, Academic medicine : journal of the Association of American Medical Colleges.

[70]  Jeroen G. W. Raaijmakers,et al.  Spacing and repetition effects in human memory: application of the SAM model , 2003, Cogn. Sci..

[71]  Robert Simon,et al.  Which Reality Matters? Questions on the Path to High Engagement in Healthcare Simulation , 2007, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[72]  Jeffrey D. Karpicke,et al.  Test-Enhanced Learning , 2006, Psychological science.

[73]  H Pashler,et al.  How persuasive is a good fit? A comment on theory testing. , 2000, Psychological review.

[74]  John R Anderson,et al.  Using a model to compute the optimal schedule of practice. , 2008, Journal of experimental psychology. Applied.

[75]  A. Glenberg Monotonic and nonmonotonic lag effects in paired-associate and recognition memory paradigms , 1976 .

[76]  R. Bjork Memory and metamemory considerations in the training of human beings. , 1994 .

[77]  Jonathan Solity,et al.  (www.interscience.wiley.com) DOI: 10.1002/acp.1066 Distributed and Massed Practice: From Laboratory to Classroom , 2022 .

[78]  John R. Anderson,et al.  Reflections of the Environment in Memory Form of the Memory Functions , 2022 .

[79]  R. Kneebone Simulation in surgical training: educational issues and practical implications , 2003, Medical education.

[80]  D. Anderson,et al.  Improving military readiness and U.S. competitiveness through dual use design and test technology , 1994, Proceedings of AUTOTESTCON '94.

[81]  Arthur C. Graesser,et al.  Organizing Instruction and Study to Improve Student Learning. IES Practice Guide. NCER 2007-2004. , 2007 .

[82]  T. Wickens On the form of the retention function : Comment on Rubin and Wenzel (1996) : A quantitative description of retention , 1998 .

[83]  Riccardo Russo,et al.  Spacing effects in recognition memory: When meaning matters , 2002 .

[84]  Harold Pashler,et al.  Optimizing distributed practice: theoretical analysis and practical implications. , 2009, Experimental psychology.

[85]  J. Korndorffer,et al.  Proficiency maintenance: impact of ongoing simulator training on laparoscopic skill retention. , 2005, Journal of the American College of Surgeons.

[86]  F. N. Dempster,et al.  The spacing effect: A case study in the failure to apply the results of psychological research. , 1988 .

[87]  R. Wisher,et al.  Staying Sharp: Retention of Military Knowledge and Skills , 1999 .

[88]  Michael J. Kahana,et al.  Effects of Spacing on Memory for Homogeneous Lists , 1993 .

[89]  Natalie B. Steinhauser,et al.  Review of Adaptive Training System Techniques , 2012 .

[90]  A. W. Melton The situation with respect to the spacing of repetitions and memory , 1970 .

[91]  Edward Vul,et al.  PSYCHOLOGICAL SCIENCE Research Article Spacing Effects in Learning A Temporal Ridgeline of Optimal Retention , 2022 .

[92]  Jeffrey D. Karpicke,et al.  Spaced retrieval: absolute spacing enhances learning regardless of relative spacing. , 2011, Journal of experimental psychology. Learning, memory, and cognition.

[93]  D. L. Hintzman Judgment of frequency versus recognition confidence: Repetition and recursive reminding , 2004, Memory & cognition.

[94]  Kelli M Taylor,et al.  The effects of overlearning and distributed practise on the retention of mathematics knowledge , 2006 .

[95]  Shana K. Carpenter,et al.  Using Spacing to Enhance Diverse Forms of Learning: Review of Recent Research and Implications for Instruction , 2012 .

[96]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[97]  D. Rubin,et al.  One Hundred Years of Forgetting : A Quantitative Description of Retention , 1996 .

[98]  Thomas C. Toppino,et al.  About Practice: Repetition, Spacing, and Abstraction , 2014 .

[99]  A. Glenberg,et al.  Component-levels theory of the effects of spacing of repetitions on recall and recognition , 1979, Memory & cognition.

[100]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[101]  Katherine A. Rawson,et al.  Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory? , 2009 .

[102]  Allen and Rosenbloom Paul S. Newell,et al.  Mechanisms of Skill Acquisition and the Law of Practice , 1993 .

[103]  Carolina E. Küpper-Tetzel Understanding the distributed practice effect : Strong effects on weak theoretical grounds , 2014 .

[104]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[105]  John R. Anderson,et al.  Implications of the ACT-R Learning Theory: No Magic Bullets , 2000 .

[106]  Katherine A. Rawson,et al.  Why does lag affect the durability of memory-based automaticity: loss of memory strength or interference? , 2013, Acta psychologica.

[107]  Jon M. Fincham,et al.  Practice and Retention: A Unifying Analysis , 1999 .

[108]  Kevin A. Gluck,et al.  A Formal Comparison of Model Variants for Performance Prediction , 2009 .

[109]  Douglas L. Hintzman,et al.  Spacing effects in picture memory , 1973, Memory & cognition.

[110]  H. Roediger,et al.  Inexpensive techniques to improve education: Applying cognitive psychology to enhance educational practice , 2012 .

[111]  A S Bregman Distribution of practice and between-trials interference. , 1967, Canadian journal of psychology.

[112]  T. Stafford,et al.  Tracing the Trajectory of Skill Learning With a Very Large Sample of Online Game Players , 2014, Psychological science.

[113]  Aaron S. Benjamin,et al.  What makes distributed practice effective? , 2010, Cognitive Psychology.