Data-Driven Proficiency Profiling

Deep Thought is a logic tutor where students practice constructing deductive logic proofs. Within Deep Thought is a data-driven mastery learning system (DDML), which calculates student proficiency based on rule scores weighted by expert-decided weights in order to assign problem sets of appropriate difficulty. In this study, we designed and tested a data-driven proficiency profiler (DDPP) method in order to calculate student proficiency without expert involvement. The DDPP determines student proficiency by comparing relevant student rule scores to previous students who behaved similarly in the tutor and successfully completed it. This method was compared to the original DDML method, proficiency based on average rule scores, and proficiency based on minimum rule scores. Our testing has shown that while the DDPP has the potential to accurately calculate student proficiency, more data is required to improve it.

[1]  Albert T. Corbett,et al.  Cognitive Tutor: Applied research in mathematics education , 2007, Psychonomic bulletin & review.

[2]  Zorica Bogdanovic,et al.  Providing Adaptivity in Moodle LMS Courses , 2012, J. Educ. Technol. Soc..

[3]  Janine van der Rijt,et al.  Teachers’ and students’ perceptions of assessments: A review and a study into the ability and accuracy of estimating the difficulty levels of assessment items , 2006 .

[4]  Stephen Fancsali,et al.  Causal Discovery with Models: Behavior, Affect, and Learning in Cognitive Tutor Algebra , 2014, EDM.

[5]  Ryan Shaun Joazeiro de Baker,et al.  New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization , 2013, AI Mag..

[6]  Emma Brunskill,et al.  The Impact on Individualizing Student Models on Necessary Practice Opportunities , 2012, EDM.

[7]  Jack Mostow,et al.  What and When do Students Learn? Fully Data-Driven Joint Estimation of Cognitive and Student Models , 2013, EDM.

[8]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[9]  Moffat Mathews,et al.  Data-Driven Misconception Discovery in Constraint-based Intelligent Tutoring Systems , 2012 .

[10]  Michael Eagle,et al.  Towards data-driven mastery learning , 2015, LAK.

[11]  Zoran Budimac,et al.  E-Learning personalization based on hybrid recommendation strategy and learning style identification , 2011, Comput. Educ..

[12]  Juan Pablo de Castro Fernández,et al.  Automatic classification of question difficulty level: Teachers' estimation vs. students' perception , 2012, 2012 Frontiers in Education Conference Proceedings.

[13]  Kenneth R. Koedinger,et al.  Data-driven Learner Modeling to Understand and Improve Online Learning: MOOCs and technology to advance learning and learning research (Ubiquity symposium) , 2014, UBIQ.