Using Learning Decomposition to Analyze Instructional Effectiveness in the ASSISTment System

A basic question of instruction is how effective it is in promoting student learning. This paper presents a study determining the relative efficacy of different instructional content by applying an educational data mining technique, learning decomposition. We use logistic regression to determine how much learning caused by different methods of presenting same skill, relative to each other. We analyze more than 60,000 performance data across 181 items from more than 2,000 students. Our results show that items are not all as effective on promoting student learning. We also did preliminary study on validating our results by comparing them with rankings from human experts. Our study demonstrates an easier and quicker approach of evaluating the quality of ITS contents than experimental studies.