Unsupervised Deep Autoencoders for Feature Extraction with Educational Data

The goal of this paper is to describe methods for automatically extracting features for student modeling from educational data, and students’ interaction-log data in particular, by training deep neural networks with unsupervised training. Several different types of autoencoder networks and structures are discussed, including deep neural networks, recurrent neural networks, variational autoencoders, convolutional neural networks, and asymmetric network structures. Autoencoder networks are trained to find lowdimensional, predictive embeddings of raw interaction-log data. These embeddings are then entered into a model as features for supervised classification tasks. We discuss the implications for training these network structures with educational data, including peculiarities that arise for interaction-log data that are not as commonly encountered in domains such as computer vision and natural language processing. Methods for evaluating the network training process are also discussed, with examples showing the importance and efficacy of visualizing neuron activations to diagnose common problems encountered during training and verifying that embedded representations of data follow desired distributions. We provide an example of how automatically extracted features can be used in a classification problem for the detection of student affect. In this example, student boredom was detected at levels above chance (area under the receiver operating characteristic curve = .673 versus .5 chance). Finally, opportunities for future work are discussed, including transfer learning and semisupervised methods.