Completion problems can reduce the illusions of understanding in a computer-based learning environment on genetics

Abstract Inaccurate judgments of task difficulty and invested mental effort may negatively affect how accurate students monitor their own performance. When students are not able to accurately monitor their own performance, they cannot control their learning effectively (e.g., allocate adequate mental effort and study time). Although students' judgments of task difficulty and invested mental effort are closely related to their study behaviors, it is still an open question how the accuracy of these judgments can be improved in learning from problem solving. The present study focused on the impact of three types of instructional support on the accuracy of students' judgments of difficulty and invested mental effort in relation to their performance while learning genetics in a computer-based environment. Sixty-seven university students with different prior knowledge received either incomplete worked-out examples, completion problems, or conventional problems. Results indicated that lower prior knowledge students performed better with completion problems, while higher prior knowledge students performed better with conventional problems. Incomplete worked-out examples resulted in an overestimation of performance, that is, an illusion of understanding, whereas completion and conventional problems showed neither over- nor underestimation. The findings suggest that completion problems can be used to avoid students' misjudgments of their competencies.

[1]  T. O. Nelson,et al.  A comparison of current measures of the accuracy of feeling-of-knowing predictions , 1984 .

[2]  Slava Kalyuga,et al.  Incorporating Learner Experience into the Design of Multimedia Instruction. , 2000 .

[3]  A. Koriat Monitoring one's own knowledge during study : A cue-utilization approach to judgments of learning , 1997 .

[4]  John Dunlosky,et al.  The Self-Monitoring Approach for Effective Learning , 2005 .

[5]  P. Pintrich,et al.  Students' motivational beliefs and their cognitive engagement in classroom academic tasks. , 1992 .

[6]  Paul Ayres Using subjective measures to detect variations of intrinsic cognitive load within problems , 2006 .

[7]  T. Gog,et al.  Effects of worked examples, example-problem, and problem-example pairs on novices learning , 2011 .

[8]  Eugene Sam Paik,et al.  Learning with Animation and the Illusion of Understanding. , 2013 .

[9]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[10]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[11]  John Dunlosky,et al.  Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention , 2012 .

[12]  John Sweller,et al.  Cognitive Load Theory , 2020, Encyclopedia of Education and Information Technologies.

[13]  Nate Kornell,et al.  Study efficacy and the region of proximal learning framework. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[14]  Martin Reisslein,et al.  Investigating the presentation and format of instructional prompts in an electrical circuit analysis computer-based learning environment , 2005, IEEE Transactions on Education.

[15]  R. Atkinson,et al.  Structuring the Transition From Example Study to Problem Solving in Cognitive Skill Acquisition: A Cognitive Load Perspective , 2003 .

[16]  Daniel L. Dinsmore,et al.  What are confidence judgments made of? Students' explanations for their confidence ratings and what that means for calibration , 2013 .

[17]  F. Paas,et al.  Instructional Efficiency: Revisiting the Original Construct in Educational Research , 2008 .

[18]  Slava Kalyuga,et al.  When problem solving is superior to studying worked examples. , 2001 .

[19]  R. Catrambone Generalizing Solution Procedures Learned From Examples , 1996 .

[20]  Douglas J. Hacker,et al.  A Comparison of the Effects of Practice Tests and Traditional Review on Performance and Calibration , 2001 .

[21]  Nancy J. Stone,et al.  Exploring the Relationship between Calibration and Self-Regulated Learning , 2000 .

[22]  Fred Paas,et al.  Effectiveness of cognitive-load based adaptive instruction in genetics education , 2011, Comput. Hum. Behav..

[23]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[24]  Cornelia S. Große,et al.  How Fading Worked Solution Steps Works – A Cognitive Load Perspective , 2004 .

[25]  T. S. Hilbert,et al.  Example-Based Learning in Heuristic Domains: A Cognitive Load Theory Account , 2009 .

[26]  Jeroen J. G. van Merriënboer,et al.  Strategies for computer-based programming instruction: Program completion vs. program generation. , 1992 .

[27]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[28]  A. Efklides Metacognition and Affect: What Can Metacognitive Experiences Tell Us about the Learning Process?. , 2006 .

[29]  K. Zabrucky,et al.  Calibration of Comprehension: Research and Implications for Education and Instruction. , 1998, Contemporary educational psychology.

[30]  Asher Koriat,et al.  The relationships between monitoring, regulation and performance , 2012 .

[31]  J. Flavell Metacognitive aspects of problem solving , 1976 .

[32]  G. Keren Calibration and probability judgements: Conceptual and methodological issues , 1991 .

[33]  T. Gog,et al.  Example-Based Learning: Integrating Cognitive and Social-Cognitive Research Perspectives , 2010 .

[34]  Fred Paas,et al.  Completion of partially worked-out examples as a generation strategy for improving monitoring accuracy , 2013 .

[35]  A. Renkl Worked-out examples: instructional explanations support learning by self- explanations , 2002 .

[36]  Fred Paas,et al.  Exploring Multidimensional Approaches to the Efficiency of Instructional Conditions , 2004 .

[37]  Colin Camerer Judgment and decision making, J. Frank Yates. Englewood Cliffs, New Jersey, Prentice-Hall inc. 1990 , 1991 .

[38]  Kerri L. Johnson,et al.  Why the Unskilled are Unaware: Further Explorations of (Absent) Self-Insight Among the Incompetent , 2006, Organizational behavior and human decision processes.

[39]  Alexander Renkl,et al.  Learning from Worked-Out-Examples: A Study on Individual Differences , 1997, Cogn. Sci..

[40]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[41]  John Sweller,et al.  Instructional Design in Technical Areas , 1999 .

[42]  K. VanLehn Problem solving and cognitive skill acquisition , 1989 .

[43]  R. Atkinson Optimizing learning from examples using animated pedagogical agents. , 2002 .

[44]  F. Paas,et al.  A motivational perspective on the relation between mental effort and performance: Optimizing learner involvement in instruction , 2005 .

[45]  M. Chi Two Approaches to the Study of Experts' Characteristics , 2006 .

[46]  Li Cao,et al.  The effect of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self-efficacy , 2006 .

[47]  Gregory Schraw,et al.  Test difficulty and judgment bias , 1994, Memory & cognition.

[48]  F. Paas,et al.  Variability of Worked Examples and Transfer of Geometrical Problem-Solving Skills: A Cognitive-Load Approach , 1994 .

[49]  Gregory Schraw A conceptual analysis of five measures of metacognitive monitoring , 2009 .

[50]  J. G. Schuurman,et al.  Redirecting learners' attention during training: Effects on cognitive load, transfer test performance and training efficiency. , 2002 .

[51]  T. Gog,et al.  Optimising worked example instruction: Different ways to increase germane cognitive load. , 2006 .

[52]  A. Koriat,et al.  Comparing objective and subjective learning curves: judgments of learning exhibit increased underconfidence with practice. , 2002, Journal of experimental psychology. General.

[53]  Slava Kalyuga,et al.  Managing split-attention and redundancy in multimedia instruction , 1999 .

[54]  A. Koriat,et al.  Processes underlying metacognitive judgments: Information-based and experience-based monitoring of one's own knowledge. , 1999 .

[55]  Richard P. Larrick,et al.  Skilled or unskilled, but still unaware of it: how perceptions of difficulty drive miscalibration in relative comparisons. , 2006, Journal of personality and social psychology.

[56]  Nate Kornell,et al.  A stability bias in human memory: overestimating remembering and underestimating learning. , 2009, Journal of experimental psychology. General.

[57]  Linda Bol,et al.  Explaining calibration accuracy in classroom contexts: the effects of incentives, reflection, and explanatory style , 2008 .

[58]  J. Sweller,et al.  Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions , 2005 .

[59]  A. Efklides Metacognitive experiences in problem solving: Metacognition, motivation, and self-regulation. , 2001 .

[60]  A. Renkl,et al.  How Effective are Instructional Explanations in Example-Based Learning? A Meta-Analytic Review , 2010 .

[61]  Slava Kalyuga Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction , 2007 .

[62]  David J. Therriault,et al.  Accuracy of metacognitive monitoring affects learning of texts. , 2003 .

[63]  T. O. Nelson,et al.  Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not of the absolute performance on an individual item. Comments on Schraw (1995) , 1996 .

[64]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[65]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[66]  Patricia A. Alexander,et al.  Calibration: What Is It and Why It Matters? An Introduction to the Special Issue on Calibrating Calibration. , 2013 .

[67]  Richard Catrambone,et al.  Aiding subgoal learning: Effects on transfer. , 1995 .

[68]  Fred Paas,et al.  Effects of studying sequences of process-oriented and product-oriented worked examples on troubleshooting transfer efficiency , 2008 .

[69]  Kristopher J Preacher,et al.  On the practice of dichotomization of quantitative variables. , 2002, Psychological methods.

[70]  T. O. Nelson,et al.  When People's Judgments of Learning (JOLs) are Extremely Accurate at Predicting Subsequent Recall: The “Delayed-JOL Effect” , 1991 .

[71]  F. Paas Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. , 1992 .

[72]  A. Hayes Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach , 2013 .

[73]  Janet Metcalfe,et al.  Judgments of Learning are Influenced by Memory for Past Test. , 2008, Journal of memory and language.

[74]  Kerri L. Johnson,et al.  Why People Fail to Recognize Their Own Incompetence , 2003 .

[75]  A. Bandura Regulation of cognitive processes through perceived self-efficacy. , 1989 .

[76]  Daniel J. Bauer,et al.  Computational Tools for Probing Interactions in Multiple Linear Regression, Multilevel Modeling, and Latent Curve Analysis , 2006 .

[77]  J. Metcalfe,et al.  A Region of Proximal Learning Model of Study Time Allocation Journal of Memory and Language , 2005 .

[78]  R. Catrambone Improving examples to improve transfer to novel problems , 1994, Memory & cognition.

[79]  R. Atkinson,et al.  Transitioning From Studying Examples to Solving Problems: Effects of Self-Explanation Prompts and Fading Worked-Out Steps. , 2003 .

[80]  Arthur M. Glenberg,et al.  Calibration of comprehension. , 1985 .

[81]  Allyson F. Hadwin,et al.  Calibration in goal setting: Examining the nature of judgments of confidence , 2013 .

[82]  A. Renkl Learning mathematics from worked-out examples: Analyzing and fostering self-explanations , 1999 .

[83]  W. Schnotz Reanalyzing the expertise reversal effect , 2010 .

[84]  David Dunning,et al.  Unskilled and unaware--but why? A reply to Krueger and Mueller (2002). , 2002, Journal of personality and social psychology.

[85]  D. Leutner,et al.  Direct Measurement of Cognitive Load in Multimedia Learning , 2003 .

[86]  S. Derry,et al.  Learning from Examples: Instructional Principles from the Worked Examples Research , 2000 .