Data-Driven Student Knowledge Assessment through Ill-Defined Procedural Tasks

The Item Response Theory (IRT) is a statistical mechanism successfully used since the beginning of the 20th century to infer student knowledge through tests. Nevertheless, existing well-founded techniques to assess procedural tasks are generally complex and applied to well-defined tasks. In this paper, we describe how, using a set of techniques we have developed based on IRT, it is possible to infer declarative student knowledge through procedural tasks. We describe how these techniques have been used with undergraduate students, in the object oriented programming domain, through ill-defined procedural exercises.

[1]  Cristina Conati,et al.  Using Bayesian Networks to Manage Uncertainty in Student Modeling , 2002, User Modeling and User-Adapted Interaction.

[2]  J. Hayes,et al.  Cognitive psychology : thinking and creating , 1978 .

[3]  Antonija Mitrovic,et al.  Constraint-based knowledge representation for individualized instruction , 2006, Comput. Sci. Inf. Syst..

[4]  Eduardo Guzmán,et al.  Improving Student Performance Using Self-Assessment Tests , 2007, IEEE Intelligent Systems.

[5]  Bert Bredeweg,et al.  Student Modelling: The Key to Individualized Knowledge-Based Instruction , 2010, NATO ASI Series.

[6]  Antonija Mitrovic,et al.  Intelligent Tutors for All: The Constraint-Based Approach , 2007, IEEE Intelligent Systems.

[7]  Eduardo Guzmán,et al.  Student Knowledge Diagnosis Using Item Response Theory and Constraint-Based Modeling , 2009, AIED.

[8]  E. J. Friedman-hill,et al.  Jess, the Java expert system shell , 1997 .

[9]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[10]  Stellan Ohlsson,et al.  Constraint-Based Student Modeling , 1994 .

[11]  John R. Anderson,et al.  Cognitive Modeling and Intelligent Tutoring , 1990, Artif. Intell..

[12]  Eduardo Guzmán,et al.  A SOA-Based Framework for Constructing Problem Solving Environments , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[13]  Ricardo Conejo,et al.  Adaptive testing for hierarchical student models , 2007, User Modeling and User-Adapted Interaction.

[14]  Wim Jansen,et al.  Multilog: Multiple, Categorical Item Analysis and Test Scoring Using Item Response Theory , 1994 .

[15]  Eduardo Guzmán,et al.  A blended E-learning experience in a course of object oriented programming fundamentals , 2009, Knowl. Based Syst..

[16]  L. L. Thurstone,et al.  A method of scaling psychological and educational tests. , 1925 .

[17]  Antonija Mitrovic,et al.  Optimising ITS Behaviour with Bayesian Networks and Decision Theory , 2001 .