Modeling student learning outcomes in studying programming language course

Learning outcome assessment is of great significance in the field of traditional on-campus teaching especially on the courses of programming languages. In this work, we take advantage of the data offered by our programming assignment judge system and propose a new IRT-BKT model for estimation of learning outcome. This new framework: Item Response Theory (IRT) model that estimates students' initial knowledge status, and joins it with the discrimination and difficulty of each skill estimated to evaluate the probability of knowing a skill before training it. We then estimate parameters learn, guess, and slip probabilities of Bayesian Knowledge Tracing (BKT) Model. Using real data, we show that IRT-BKT model outperforms Item Response Theory and Bayesian Knowledge Tracing in terms of prediction accuracy.

[1]  Zachary A. Pardos,et al.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing , 2010, UMAP.

[2]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[3]  Zachary A. Pardos,et al.  KT-IDEM: introducing item difficulty to the knowledge tracing model , 2011, UMAP'11.

[4]  James H. Block,et al.  1: Mastery Learning , 1976 .

[5]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[6]  Jim Reye,et al.  Student Modelling Based on Belief Networks , 2004, Int. J. Artif. Intell. Educ..

[7]  Peter Brusilovsky,et al.  Investigating Automated Student Modeling in a Java MOOC , 2014, EDM.

[8]  Yanbo Xu,et al.  Using Item Response Theory to Refine Knowledge Tracing , 2013, EDM.

[9]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[10]  F. Lord.  A theory of test scores. , 1952 .

[11]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[12]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[13]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[14]  Jürgen Börstler,et al.  Educational Data Mining and Learning Analytics in Programming: Literature Review and Case Studies , 2015, ITiCSE-WGR.

[15]  Beverly Park Woolf,et al.  A Dynamic Mixture Model to Detect Student Motivation and Proficiency , 2006, AAAI.

[16]  Ivon Arroyo,et al.  Tracing Knowledge and Engagement in Parallel in an Intelligent Tutoring System , 2014, EDM.

[17]  Peter Hubwieser,et al.  Evaluation of Source Code with Item Response Theory , 2015, ITiCSE.

[18]  Neil T. Heffernan,et al.  The Student Skill Model , 2012, ITS.

[19]  Erik L. L. Sonnhammer,et al.  A Hidden Markov Model for Predicting Transmembrane Helices in Protein Sequences , 1998, ISMB.

[20]  Mark D. Reckase,et al.  Item Response Theory: Parameter Estimation Techniques , 1998 .

[21]  Yanbo Xu,et al.  Using Logistic Regression to Trace Multiple Sub-skills in a Dynamic Bayes Net , 2011, EDM.

[22]  Neil T. Heffernan,et al.  The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching , 2014, International Journal of Artificial Intelligence in Education.

[23]  De Ayala.  Fundamentals of Item Response Theory [Book Review]. , 1993 .

[24]  Jussi Kasurinen,et al.  Estimating programming knowledge with Bayesian knowledge tracing , 2009, ITiCSE.

[25]  Brian W. Junker,et al.  Applications and Extensions of MCMC in IRT: Multiple Item Types, Missing Data, and Rated Responses , 1999 .

[26]  Antonija Mitrovic,et al.  Optimising ITS Behaviour with Bayesian Networks and Decision Theory , 2001 .

[27]  Leon E. Winslow,et al.  Programming pedagogy—a psychological overview , 1996, SGCS.

[28]  Kenneth R. Koedinger,et al.  Avoiding Problem Selection Thrashing with Conjunctive Knowledge Tracing , 2011, EDM.

[29]  Jeffrey A Douglas,et al.  Higher-order latent trait models for cognitive diagnosis , 2004 .

[30]  Andrew D. Martin,et al.  Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999 , 2002, Political Analysis.