Towards Suggesting Actionable Interventions for Wheel Spinning Students

In some computerized educational systems, there is evidence of students wheel-spinning, where a student tries and repeatedly fails at an educational task for learning a skill. This may be particularly concerning in low resource settings. Prior research has focused on predicting and modeling wheel-spinning, but there has been little work on how to best help students stuck in wheel-spinning. We use past student system interaction data and a minimal amount of expert input to automatically inform individualized interventions, without needing experts to label a large dataset of interventions. Our method trains a model to predict wheelspinning and utilizes a popular tool in interpretable machine learning, Shapley values, to provide individualized credit attribution over the features of the model, including actionable features like possible gaps in prerequisites. In simulation on two different statistical student models, our approach can identify a correct intervention with over 80% accuracy before the simulated student begins the activity they will wheel spin on. In our real dataset we show initial qualitative results that our proposed interventions match what an expert would prescribe.

[1]  Vincent Aleven,et al.  Early Detection of Wheel Spinning: Comparison across Tutors, Models, Features, and Operationalizations , 2019, EDM.

[2]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[3]  Joseph E. Beck,et al.  Understanding Wheel Spinning in the Context of Affective Factors , 2014, Int. J. People Oriented Program..

[4]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[5]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[6]  H. E. Stubbé,et al.  Can’t Wait to Learn: A quasi-experimental mixed-methods evaluation of a digital game-based learning programme for out-of-school children in Sudan , 2020 .

[7]  Jennifer J. Vogel-Walcutt,et al.  The Definition, Assessment, and Mitigation of State Boredom Within Educational Settings: A Comprehensive Review , 2012 .

[8]  Yue-Jun Zhang,et al.  Regional allocation of carbon emission quotas in China: Evidence from the Shapley value method , 2014 .

[9]  Erik Strumbelj,et al.  Explaining prediction models and individual predictions with feature contributions , 2014, Knowledge and Information Systems.

[10]  Sanjay Chandrasekaran,et al.  How quickly can wheel spinning be detected? , 2016, EDM.

[11]  Neil T. Heffernan,et al.  Decision Tree Modeling of Wheel-Spinning and Productive Persistence in Skill Builders. , 2018 .

[12]  Guillaume Chanel,et al.  Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[13]  L. Shapley A Value for n-person Games , 1988 .

[14]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[15]  Albert T. Corbett,et al.  Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments , 2008 .

[16]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[17]  Scott M. Lundberg,et al.  Consistent Individualized Feature Attribution for Tree Ensembles , 2018, ArXiv.

[18]  Juan Miguel L. Andres The Incidence and Persistence of Affective States While Playing Newton ’ s Playground , 2014 .

[19]  Y. Narahari,et al.  A Shapley Value-Based Approach to Discover Influential Nodes in Social Networks , 2011, IEEE Transactions on Automation Science and Engineering.

[20]  Scott M. Lundberg,et al.  Explainable machine-learning predictions for the prevention of hypoxaemia during surgery , 2018, Nature Biomedical Engineering.

[21]  Yue Gong,et al.  Wheel-Spinning: Students Who Fail to Master a Skill , 2013, AIED.

[22]  Joseph E. Beck,et al.  Considering the Influence of Prerequisite Performance on Wheel Spinning , 2015, EDM.

[23]  Ma. Mercedes T. Rodrigo,et al.  Dynamics of Student Cognitive-Affective Transitions During a Mathematics Game , 2011 .

[24]  Yue Gong,et al.  Towards Detecting Wheel-Spinning: Future Failure in Mastery Learning , 2015, L@S.