Markov Decision Process for MOOC Users Behavioral Inference

Studies on massive open online courses (MOOCs) users discuss the existence of typical profiles and their impact on the learning process of the students. However defining the typical behaviors as well as classifying the users accordingly is a difficult task. In this paper we suggest two methods to model MOOC users behaviour given their log data. We mold their behavior into a Markov Decision Process framework. We associate the user's intentions with the MDP reward and argue that this allows us to classify them.

[1]  Michael I. Jordan,et al.  Nonparametric Bayesian Learning of Switching Linear Dynamical Systems , 2008, NIPS.

[2]  Eyal Amir,et al.  Bayesian Inverse Reinforcement Learning , 2007, IJCAI.

[3]  Michael L. Littman,et al.  Apprenticeship Learning About Multiple Intentions , 2011, ICML.

[4]  Linda Corrin,et al.  Visualizing patterns of student engagement and performance in MOOCs , 2014, LAK.

[5]  Markus Wulfmeier,et al.  Maximum Entropy Deep Inverse Reinforcement Learning , 2015, 1507.04888.

[6]  Lise Getoor,et al.  Modeling Learner Engagement in MOOCs using Probabilistic Soft Logic , 2013 .

[7]  Jonathan P. How,et al.  Improving the efficiency of Bayesian inverse reinforcement learning , 2012, 2012 IEEE International Conference on Robotics and Automation.

[8]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .

[9]  Stephanie D. Teasley,et al.  Towards A General Method for Building Predictive Models of Learner Success using Educational Time Series Data , 2014, LAK Workshops.

[10]  Sebastián Ventura,et al.  Educational data science in massive open online courses , 2016, WIREs Data Mining Knowl. Discov..

[11]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[12]  ChengXiang Zhai,et al.  Modeling MOOC Student Behavior With Two-Layer Hidden Markov Models , 2017, EDM.

[13]  Sergey Levine,et al.  Nonlinear Inverse Reinforcement Learning with Gaussian Processes , 2011, NIPS.

[14]  Amit Surana,et al.  Bayesian Nonparametric Inverse Reinforcement Learning for Switched Markov Decision Processes , 2014, 2014 13th International Conference on Machine Learning and Applications.

[15]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[16]  Firas Jarboui,et al.  Users Behavioural Inference with Markovian Decision Process and Active Learning , 2017, IAL@PKDD/ECML.

[17]  Gautam Biswas,et al.  Early Prediction of Student Dropout and Performance in MOOCs using Higher Granularity Temporal Information , 2014, J. Learn. Anal..

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.