The Effect of Prior Conceptual Knowledge on Procedural Performance and Learning in Algebra

The Effect of Prior Conceptual Knowledge on Procedural Performance and Learning in Algebra Julie L. Booth (juliebooth@cmu.edu) Kenneth R. Koedinger (koedinger@cmu.edu) Human Computer Interaction Institute, Carnegie Mellon University Pittsburgh, PA 15213 USA Robert S. Siegler (rs7k@andrew.cmu.edu) Department of Psychology, Carnegie Mellon University Pittsburgh, PA 15213 USA Keywords: Psychology; Education; concepts and categories; learning; problem-solving; human experimentation Introduction Errors are inevitable when individuals are first learning any skill; solving algebraic equations is no exception. Students often use incorrect procedures when learning Algebra (e.g., Lerch, 2004), and use of incorrect procedures has been hypothesized to be attributable to misunderstandings or gaps in students’ conceptual knowledge of Algebra (Anderson, 1989; Van Lehn & Jones, 1993). Experiment 1 in the current study tests this hypothesis; Experiment 2 examines whether these pretest misconceptions also affect learning of algebraic problem- solving Methods In Experiment 1, 112 middle and high school students using the Algebra I Cognitive Tutor, a self-paced intelligent tutor system (Koedinger, Anderson, Hadley, & Mark, 1997), completed a test assessing their conceptual and procedural knowledge of algebra before beginning the unit on solving two-step linear equations; there were two forms of the test and half of the students were randomly assigned to receive each. In Experiment 2, the 97 students who completed the tutor unit took the opposite form of the test. Results and Discussion Students in Experiment 1who had a good concept of the equals sign made equals sign-related errors (e.g., performing operations to only one side of an equation, dropping the equals sign from the equation) on fewer problems (10%) than those with a poor concept of the equal sign (21%; t(110) = 2.07, p < .05). Similarly, those with a good concept of like terms combined unlike terms in fewer problems (1%) than those with poorer knowledge of like terms (9%; t(110) = 2.59, p < .01). Lack of conceptual knowledge of certain features was also correlated with fewer procedural problems solved correctly (negatives (R(112) =.38, p < .01), like terms (R(112) =.32, p < .01) and equals sign (R(112) =.59, p < .01); lack of equals sign knowledge is predictive of performance beyond the other two misconceptions (21% variance added; t(109) = 5.95, p < .01) In Experiment 2, improved equals sign knowledge correlated with improvement in problem-solving (controlling for the amount of procedural improvement possible), (partial R(97) = .23, p < .05). In addition, two marginally significant trends were found suggesting that pretest knowledge of negatives and of like terms predicted students’ learning on transfer problems (partial Rs(97) = .17, ps < .10). Results from this study indicate that a lack of knowledge about certain conceptual features is associated with use of related incorrect procedures when solving equations. These misconceptions lead students to solve less problems correctly and, in some cases, to learn less from instruction unless they are corrected. This suggests that improving students’ knowledge of the conceptual features that underlie Algebra may be necessary for robust learning to occur. Acknowledgments Funding for this research is provided by the National Science Foundation, Grant Number SBE-0354420 to the Pittsburgh Science of Learning Center (PSLC, http://www.learnlab.org). References Anderson, J.R. (1989). The analogical origins of errors in problem solving. In D. Klahr & K. Kotovsky (Eds). Complex information processing: The impact of Herbert A. Simon. Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc. Koedinger, K.R., Anderson, J.R., Hadley, W.H., & Mark, M.A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43. Lerch, C. M. (2004). Control decisions and personal beliefs: Their effect on solving mathematical problems. Journal of Mathematical Behavior, 23, 21-36. Van Lehn, K., & Jones, R.M. (1993). What mediates the self-explanation effect? Knowledge gaps, schemas, or analogies? In M. Polson (Ed.) Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society (pp. 1034-1039).

[1]  Anna N. Rafferty,et al.  Applying Learning Factors Analysis to Build Stereotypic Student Models , 2007, AIED.

[2]  K. Squire,et al.  Design-Based Research: Putting a Stake in the Ground , 2004 .

[3]  A. diSessa Toward an Epistemology of Physics , 1993 .

[4]  D. Wood,et al.  Help seeking, learning and contingent tutoring , 1999, Comput. Educ..

[5]  Ryan Shaun Joazeiro de Baker,et al.  Detecting When Students Game the System, Across Tutor Subjects and Classroom Cohorts , 2005, User Modeling.

[6]  Kenneth R. Koedinger,et al.  Is Over Practice Necessary? - Improving Learning Efficiency with the Cognitive Tutor through Educational Data Mining , 2007, AIED.

[7]  Jack Mostow,et al.  A La Recherche du Temps Perdu, or As Time Goes By: Where Does the Time Go in a Reading Tutor That Listens? , 2002, Intelligent Tutoring Systems.

[8]  K. VanLehn,et al.  Self-explaining in the Classroom: Learning Curve Evidence , 2007 .

[9]  Ryan Shaun Joazeiro de Baker,et al.  Adapting to When Students Game an Intelligent Tutoring System , 2006, Intelligent Tutoring Systems.

[10]  Ryan Shaun Joazeiro de Baker,et al.  Detecting Student Misuse of Intelligent Tutoring Systems , 2004, Intelligent Tutoring Systems.

[11]  Philip I. Pavlik Optimizing Knowledge Component Learning Using a Dynamic Structural Model of Practice , 2007 .

[12]  Kenneth R. Koedinger,et al.  Can a Polite Intelligent Tutoring System Lead to Improved Learning Outside of the Lab? , 2007, AIED.

[13]  Kurt VanLehn,et al.  Effects of Dissuading Unnecessary Help Requests While Providing Proactive Help , 2005, AIED.

[14]  Ronald H. Stevens,et al.  Modeling the Development of Problem Solving Skills in Chemistry with a Web-Based Tutor , 2004, Intelligent Tutoring Systems.

[15]  Jack Mostow,et al.  Automatically Assessing Oral Reading Fluency in a Computer Tutor that Listens , 2004 .

[16]  Kenneth R. Koedinger,et al.  An architecture for plug-in tutor agents , 1996 .

[17]  Tim Penelitian dan Pengembangan Wahana Komputer Microsoft office 97 , 2001 .

[18]  Beverly Park Woolf,et al.  Repairing Disengagement With Non-Invasive Interventions , 2007, AIED.

[19]  E. Maris Psychometric latent response models , 1995 .

[20]  Beverly Park Woolf,et al.  Inferring learning and attitudes from a Bayesian Network of log file data , 2005, AIED.

[21]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[22]  Randolph M. Jones,et al.  What Mediates the Self-explanation Eeect? Knowledge Gaps, Schemas or Analogies? , 1993 .

[23]  Carol M. Lerch Control decisions and personal beliefs: their effect on solving mathematical problems , 2004 .

[24]  Neil T. Heffernan,et al.  Informing Teachers Live about Student Learning: Reporting in Assistment System , 2005 .

[25]  Richard C. Anderson,et al.  FEEDBACK PROCEDURES IN COMPUTER-ASSISTED ARITHMETIC INSTRUCTION , 1973 .

[26]  Antonija Mitrovic,et al.  Constraint-based knowledge representation for individualized instruction , 2006, Comput. Sci. Inf. Syst..

[27]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[28]  Ryan Shaun Joazeiro de Baker,et al.  Do Performance Goals Lead Students to Game the System? , 2005, AIED.

[29]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[30]  Vincent Aleven,et al.  Toward Tutoring Help Seeking: Applying Cognitive Modeling to Meta-cognitive Skills , 2004, Intelligent Tutoring Systems.

[31]  Julita Vassileva,et al.  Adaptive Reward Mechanism for Sustainable Online Learning Community , 2005, AIED.

[32]  Mathieu Koppen,et al.  Introduction to knowledge spaces: How to build, test, and search them , 1990 .

[33]  Joseph E. Beck,et al.  Engagement tracing: using response times to model student disengagement , 2005, AIED.

[34]  K. Tatsuoka RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[35]  Neil T. Heffernan,et al.  Addressing the testing challenge with a web-based e-assessment system that tutors as it assesses , 2006, WWW '06.

[36]  Paul R. Cohen,et al.  Modeling learning patterns of students with a tutoring system using Hidden Markov Models , 2007, AIED.

[37]  Kurt VanLehn,et al.  Exploring Alternative Methods for Error Attribution in Learning Curves Analysis in Intelligent Tutoring Systems , 2007, AIED.

[38]  Neil T. Heffernan,et al.  Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[39]  John R. Anderson The Analogical Origins of Errors in Problem Solving , 1989 .

[40]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.