The Knowledge-Learning-Instruction (KLI) Framework: Toward Bridging the Science-Practice Chasm to Enhance Robust Student Learning

Executive Summary The volume of research on learning and instruction is enormous. Yet progress in improving educational outcomes has been slow at best. Many learning science results have not been translated into general practice and it appears that most that have been fielded have not yielded significant results in randomized control trials. Addressing the chasm between learning science and educational practice will require massive efforts from many constituencies, but one of these efforts is to develop a theoretical framework that permits a more systematic accumulation of the relevant research base.

[1]  J. Greeno THE SITUATIVITY OF KNOWING, LEARNING, AND RESEARCH , 1998 .

[2]  Vincent Aleven,et al.  Worked Examples and Tutored Problem Solving: Redundant or Synergistic Forms of Support? , 2009, Top. Cogn. Sci..

[3]  Stephen M. Fiore,et al.  At a Loss From Words: Verbal Overshadowing of Perceptual Memories , 1997 .

[4]  I. Biederman,et al.  Sexing Day-Old Chicks : A Case Study and Expert Systems Analysis of a Difficult Perceptual-Learning Task , 1987 .

[5]  S. Michaels,et al.  Deliberative Discourse Idealized and Realized: Accountable Talk in the Classroom and in Civic Life , 2008 .

[6]  B. Bloom Taxonomy of educational objectives , 1956 .

[7]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[8]  Charles H. Shea,et al.  Contextual interference: Contributions of practice , 1990 .

[9]  Vincent Aleven,et al.  An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor , 2002, Cogn. Sci..

[10]  Ron Sun,et al.  Integrating rules and connectionism for robust commonsense reasoning , 1994, Sixth-generation computer technology series.

[11]  John R. Anderson,et al.  Illustrating Principled Design: The Early Evolution of a Cognitive Tutor for Algebra Symbolization , 1998, Interact. Learn. Environ..

[12]  Alfred Bork,et al.  Multimedia in Learning , 2001 .

[13]  Catherine E. Snow,et al.  Preventing reading difficulties in young children , 1998 .

[14]  Philip I. Pavlik,et al.  Understanding and applying the dynamics of test practice and study practice , 2007 .

[15]  Marcus Taft,et al.  Using radicals in teaching Chinese characters to second language learners , 1999 .

[16]  Charles A. Perfetti,et al.  Chinese Semantic Radicals 1 Running head : CHINESE SEMANTIC RADICALS Learning Vocabulary in Chinese as a Foreign Language : Effects of Explicit Instruction and Semantic Cue Reliability , 2011 .

[17]  J. Sweller,et al.  Structuring Effective Worked Examples , 1990 .

[18]  Lisa Anthony,et al.  Developing Handwriting-based Intelligent Tutors to Enhance Mathematics Learning , 2008 .

[19]  Alexander Renkl,et al.  Learning from Worked-Out-Examples: A Study on Individual Differences , 1997, Cogn. Sci..

[20]  Kurt VanLehn,et al.  Explaining Self-Explaining: A Contrast between Content and Generation , 2007, AIED.

[21]  Jennifer Wiley,et al.  Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. , 1999 .

[22]  C. Shea,et al.  Principles derived from the study of simple skills do not generalize to complex skill learning , 2002, Psychonomic bulletin & review.

[23]  Andreas J. Stylianides,et al.  Facilitating the Transition from Empirical Arguments to Proof. , 2009 .

[24]  P. Chandler,et al.  Why Some Material Is Difficult to Learn , 1994 .

[25]  K. McRae,et al.  Proceedings of the 30th Annual Conference of the Cognitive Science Society. , 2008 .

[26]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[27]  Deborah Loewenberg Ball,et al.  Understanding and describing mathematical knowledge for teaching: knowledge about proof for engaging students in the activity of proving , 2008 .

[28]  J. Sweller,et al.  Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions , 2005 .

[29]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[30]  R. Mayer,et al.  Animations need narrations : an experimental test of a dual-coding hypothesis , 1991 .

[31]  P. Skudlarski,et al.  Development of left occipitotemporal systems for skilled reading in children after a phonologically- based intervention , 2004, Biological Psychiatry.

[32]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[33]  J. von Neumann,et al.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[34]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[35]  Robbie Case,et al.  The Role of Central Conceptual Structures in the Development of Children's Thought , 1995 .

[36]  K. VanLehn,et al.  Scaffolding Deep Comprehension Strategies Through Point&Query, AutoTutor, and iSTART , 2005 .

[37]  Keith J Holyoak,et al.  Pragmatic reasoning schemas , 1985, Cognitive Psychology.

[38]  B. Bloom,et al.  Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , 1966 .

[39]  Sandra Katz,et al.  Towards the Design of More Effective Advisors for Learning-by-Doing Systems , 1996, Intelligent Tutoring Systems.

[40]  Kenneth R. Koedinger,et al.  Trade-Offs Between Grounded and Abstract Representations: Evidence From Algebra Problem Solving , 2008, Cogn. Sci..

[41]  Slava Kalyuga,et al.  Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning , 2005 .

[42]  Yoram Singer,et al.  Unsupervised Models for Named Entity Classification , 1999, EMNLP.

[43]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[44]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[45]  Douglas L. Medin,et al.  Context theory of classification learning. , 1978 .

[46]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[47]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[48]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[49]  Kenneth R. Koedinger,et al.  Is Over Practice Necessary? - Improving Learning Efficiency with the Cognitive Tutor through Educational Data Mining , 2007, AIED.

[50]  R. Atkinson,et al.  Transitioning From Studying Examples to Solving Problems: Effects of Self-Explanation Prompts and Fading Worked-Out Steps. , 2003 .

[51]  John R. Anderson,et al.  Implications of the ACT-R Learning Theory: No Magic Bullets , 2000 .

[52]  Baruch B. Schwarz,et al.  The Effects of Monological and Dialogical Argumentation on Concept Learning in Evolutionary Theory , 2007 .

[53]  Tom M. Mitchell,et al.  Explanation-Based Generalization: A Unifying View , 1986, Machine Learning.

[54]  William M. Carroll Using worked examples as an instructional support in the algebra classroom. , 1994 .

[55]  F. Paas Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. , 1992 .

[56]  Vincent Aleven,et al.  The worked-example effect: Not an artefact of lousy control conditions , 2009, Comput. Hum. Behav..

[57]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[58]  B. Rittle-Johnson,et al.  Developing Conceptual Understanding and Procedural Skill in Mathematics: An Iterative Process. , 2001 .

[59]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[60]  Daniel L. Schwartz,et al.  Rethinking transfer: A simple proposal with multiple implica-tions , 1999 .

[61]  R. A. Tarmizi,et al.  Guidance during Mathematical Problem Solving. , 1988 .

[62]  John R. Anderson,et al.  Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes , 2001, CHI.

[63]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[64]  K. Koedinger,et al.  In Vivo Experiments on Whether Supporting Metacognition in Intelligent Tutoring Systems Yields Robust Learning , 2009 .

[65]  Kurt VanLehn,et al.  A model of the self-explanation effect. , 1992 .

[66]  H A Simon,et al.  Cue recognition and cue elaboration in learning from examples. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[67]  Slava Kalyuga,et al.  When problem solving is superior to studying worked examples. , 2001 .

[68]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[69]  J. Sweller,et al.  Effects of schema acquisition and rule automation on mathematical problem-solving transfer. , 1987 .

[70]  J. Sweller,et al.  The Use of Worked Examples as a Substitute for Problem Solving in Learning Algebra , 1985 .

[71]  James L. McClelland,et al.  Teaching the /r/–/l/ discrimination to Japanese adults: behavioral and neural aspects , 2002, Physiology & Behavior.

[72]  A. Elliot A Conceptual History of the Achievement Goal Construct. , 2005 .

[73]  Robert L. Goldstone,et al.  The Transfer of Scientific Principles Using Concrete and Idealized Simulations , 2005, Journal of the Learning Sciences.

[74]  Randolph M. Jones,et al.  Cascade Explains and Informs the Utility of Fading Examples to Problems , 2001 .

[75]  F. Gobet,et al.  Expertise, models of learning and computer-based tutoring , 1999, Comput. Educ..

[76]  Mitchell J. Nathan,et al.  The Real Story Behind Story Problems: Effects of Representations on Quantitative Reasoning , 2004 .

[77]  Kenneth R. Koedinger,et al.  The developmental progression from implicit to explicit knowledge: A computational approach , 1999 .

[78]  David Klahr,et al.  Dual Space Search During Scientific Reasoning , 1988, Cogn. Sci..

[79]  Slava Kalyuga,et al.  Measuring Knowledge to Optimize Cognitive Load Factors During Instruction. , 2004 .

[80]  John R. Anderson,et al.  Learning Artificial Grammars With Competitive Chunking , 1990 .

[81]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[82]  Cornelia S. Große,et al.  How Fading Worked Solution Steps Works – A Cognitive Load Perspective , 2004 .

[83]  Baruch B. Schwarz,et al.  The role of argumentation and explanation in conceptual change: Indications from protocol analyses of peer-to-peer dialogue , 2009 .

[84]  K. Holyoak,et al.  A symbolic-connectionist theory of relational inference and generalization. , 2003, Psychological review.

[85]  Arthur C. Graesser,et al.  Organizing Instruction and Study to Improve Student Learning. IES Practice Guide. NCER 2007-2004. , 2007 .

[86]  Fernand Gobet,et al.  Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/acp.1110 Chunking Models of Expertise: Implications for Education FERNAND GOBET* , 2022 .

[87]  Kenneth R. Koedinger,et al.  When and how often should worked examples be given to students? New results and a summary of the current state of research , 2008 .

[88]  M. Levine Hypothesis behavior by humans during discrimination learning. , 1966, Journal of experimental psychology.

[89]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[90]  F. Paas,et al.  Variability of Worked Examples and Transfer of Geometrical Problem-Solving Skills: A Cognitive-Load Approach , 1994 .

[91]  Alexander Renkl,et al.  From Studying Examples to Solving Problem: Fading Worked-Out Solution Steps Helps Learning , 2000 .

[92]  H. Mandl,et al.  Learning from Worked-Out Examples: The Effects of Example Variability and Elicited Self-Explanations , 1998, Contemporary educational psychology.

[93]  Ann L. Brown,et al.  Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities , 1984 .

[94]  J. G. Schuurman,et al.  Redirecting learners' attention during training: Effects on cognitive load, transfer test performance and training efficiency. , 2002 .

[95]  K. Koedinger,et al.  Exploring the Assistance Dilemma in Experiments with Cognitive Tutors , 2007 .

[96]  I. Biederman,et al.  Sexing day-old chicks: A case study and expert systems analysis of a difficult perceptual-learning task. , 1987 .

[97]  Clayton Lewis,et al.  Why and How to Learn Why: Analysis-Based Generalization of Procedures , 1988, Cogn. Sci..

[98]  R. Schmidt,et al.  New Conceptualizations of Practice: Common Principles in Three Paradigms Suggest New Concepts for Training , 1992 .

[99]  Steve Graham Inaugural Editorial for Journal of Educational Psychology , 2009 .

[100]  R. Catrambone Generalizing Solution Procedures Learned From Examples , 1996 .

[101]  R. Harald Baayen,et al.  Predicting the dative alternation , 2007 .

[102]  John R. Anderson The Adaptive Character of Thought , 1990 .

[103]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[104]  A. diSessa Toward an Epistemology of Physics , 1993 .

[105]  Richard K. Staley,et al.  From Example Study to Problem Solving: Smooth Transitions Help Learning , 2002 .

[106]  David F. Feldon,et al.  Cognitive task analysis , 2009 .

[107]  R Todd Constable,et al.  The neurobiology of adaptive learning in reading: A contrast of different training conditions , 2004, Cognitive, affective & behavioral neuroscience.

[108]  S. Ainsworth,et al.  Multiple Forms of Dynamic Representation. , 2004 .

[109]  Slava Kalyuga,et al.  Learner Experience and Efficiency of Instructional Guidance , 2001 .

[110]  Kenneth R. Koedinger,et al.  The impact of spurious correlations on students' problem-solving , 2004 .

[111]  Y. Lincoln,et al.  Scientific Research in Education , 2004 .

[112]  Craig A. Knoblock,et al.  Active + Semi-supervised Learning = Robust Multi-View Learning , 2002, ICML.

[113]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[114]  Dedre Gentner,et al.  Structure-Mapping: A Theoretical Framework for Analogy , 1983, Cogn. Sci..

[115]  G. Hanley e‐Learning and the Science of Instruction , 2004 .

[116]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[117]  J. Sweller COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[118]  Z. Dienes,et al.  A theory of implicit and explicit knowledge , 1999, Behavioral and Brain Sciences.

[119]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[120]  Jill L. Quilici,et al.  Role of examples in how students learn to categorize statistics word problems. , 1996 .

[121]  Kenneth R. Koedinger,et al.  Key Misconceptions in Algebraic Problem Solving , 2008 .

[122]  Randolph M. Jones,et al.  Why Example Fading Works: A Qualitative Analysis Using Cascade , 2019, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society.

[123]  James L. McClelland,et al.  Parallel Distributed Processing: Bridging the gap between human and machine intelligence , 1990 .

[124]  Shaaron Ainsworth,et al.  The effects of self-explaining when learning with text or diagrams , 2003, Cogn. Sci..

[125]  Kenneth R. Koedinger,et al.  Seeing language learning inside the math: Cognitive analysis yields transfer , 2010 .

[126]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..