Analysis of Student Behaviour in Habitable Worlds Using Continuous Representation Visualization

We introduce a novel approach to visualizing temporal clickstream behaviour in the context of a degree-satisfying online course, Habitable Worlds, offered through Arizona State University. The current practice for visualizing behaviour within a digital learning environment has been to utilize state space graphs and other plots of descriptive statistics on resource transitions. While these forms can be visually engaging, they rely on conditional frequency tabulations which lack contextual depth and require assumptions about the patterns being sought. Skip-grams and other representation learning techniques position elements into a vector space which can capture a wide scope of regularities in the data. These regularities can then be projected onto a two-dimensional perceptual space using dimensionality reduction techniques designed to retain relationships information encoded in the learned representations. While these visualization techniques have been used before in the broader machine learning community to better understand the makeup of a neural network hidden layer or the relationship between word vectors, we apply them to online behavioral learner data and go a step further; exploring the impact of the parameters of the model on producing tangible, non-trivial observations of behaviour that are illuminating and suggestive of pedagogical improvement to the course designers and instructors. The methodology introduced in this paper led to an improved understanding of passing and non-passing student behavior in the course and is widely applicable to other datasets of clickstream activity where investigators and stakeholders wish to organically surface principal behavioral patterns.