Methods to find the number of latent skills

Identifying the skills that determine the success or failure to exercises and question items is a dicult task. Multiple skills may be involved at various degree of importance, and skills may overlap and correlate. In an eort towards the goal of nding the skills behind a set of items, we investigate two techniques to determine the number of dominant latent skills. The Singular Value Decomposition (SVD) is a known technique to nd latent factors. The singular values represent direct evidence of the strength of latent factors. Application of SVD to nding the number of latent skills is explored. We introduce a second technique based on a wrapper approach. Linear models with dierent number of skills are built, and the one that yields the best prediction accuracy through cross validation is considered the most appropriate. The results show that both techniques are eective in identifying the latent factors over synthetic data. An investigation with real data from the fraction algebra domain is also reported. Both the SVD and wrapper methods yield results that have no simple interpretation.

[1]  Jirí Vomlel,et al.  Bayesian Networks In Educational Testing , 2002, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[2]  Michel C. Desmarais,et al.  Item to Skills Mapping: Deriving a Conjunctive Q-matrix from Data , 2012, ITS.

[3]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[4]  Michel C. Desmarais Conditions for Effectively Deriving a Q-Matrix from Data with Non-negative Matrix Factorization. Best Paper Award , 2011, EDM.

[5]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[6]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[7]  Kenneth R. Koedinger,et al.  Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work , 2007 .

[8]  Renaud Gaujoux,et al.  A flexible R package for nonnegative matrix factorization , 2010, BMC Bioinformatics.

[9]  Tanja Mitrovic,et al.  Constraint-based tutors: a success story , 2001, AIED.

[10]  Tiffany Barnes,et al.  The Q-matrix Method: Mining Student Response Data for Knowledge , 2005 .

[11]  Author,et al.  Item to skills mapping : Deriving a conjunctive Q-matrix from data , 2012 .

[12]  Christian R. Shelton,et al.  Investigating Generative Factors of Score Matrices , 2007, AIED.

[13]  Antonija Mitrovic,et al.  A Knowledge Acquisition System for Constraint-based Intelligent Tutoring Systems , 2005, AIED.