Exploring User Data From a Game-like Math Tutor: A Case Study in Causal Modeling

We have used causal modeling to understand data from a game-like math tutor, Monkey’s Revenge. We collected student data of various types such as their attitude and enjoyment via surveys, performance within tutor via logging, and learning as measured by a pre/post test. Although the data are observational, we want to understand the causal relationships between the variables we have collected. We contrast the causal modeling approach to the results we achieve with traditional approaches such as correlation matrix and multiple regression. Relative to traditional approaches, we found that causal modeling did a better job at detecting and representing spurious association, and direct and indirect effects. We found that the causal model, particularly one augmented with domain knowledge about likely causal relationships, resulted in much more plausible and interpretable model. We present a case study for blending exploratory results from causal modeling with randomized controlled studies to validate hypotheses.