Connectionist Recommendation in the Wild On the utility , scrutability , and scalability of neural networks for personalized course guidance

The aggregate behaviors of users can collectively encode deep semantic information about the objects with which they interact. In this paper, we demonstrate novel ways in which the synthesis of these data can illuminate the terrain of users' environment and support them in their decision making and wayfinding. A novel application of Recurrent Neural Networks and skip-gram models, approaches popularized by their application to modeling language, are brought to bear on student university enrollment sequences to create vector representations of courses and map out traversals across them. We present demonstrations of how scrutability from these neural networks can be gained and how the combination of these techniques can be seen as an evolution of content tagging and a means for a recommender to balance user preferences inferred from data with those explicitly specified. From validation of the models to the development of a UI, we discuss additional requisite functionality informed by the results of a field study leading to the ultimate deployment of the system at a university.

[1]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[2]  Chris Piech,et al.  Learning to Represent Student Knowledge on Programming Exercises Using Deep Learning , 2017, EDM.

[3]  Omer Levy,et al.  Neural Word Embedding as Implicit Matrix Factorization , 2014, NIPS.

[4]  ManolopoulosYannis,et al.  Collaborative recommender systems , 2008 .

[5]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[6]  Nemanja Djuric,et al.  E-commerce in Your Inbox: Product Recommendations at Scale , 2015, KDD.

[7]  B. Zimmerman Self-Regulated Learning and Academic Achievement: An Overview , 1990 .

[8]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[9]  Albert T. Corbett,et al.  Cognitive Computer Tutors: Solving the Two-Sigma Problem , 2001, User Modeling.

[10]  Yehuda Koren,et al.  Advances in Collaborative Filtering , 2011, Recommender Systems Handbook.

[11]  Oren Barkan,et al.  ITEM2VEC: Neural item embedding for collaborative filtering , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[12]  Eitel J. M. Lauría,et al.  Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative , 2014, J. Learn. Anal..

[13]  Xiangnan He,et al.  A Generic Coordinate Descent Framework for Learning from Implicit Feedback , 2016, WWW.

[14]  Christopher Mazzeo,et al.  Improving Credit Mobility for Community College Transfer Students , 2016 .

[15]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[16]  Judy Kay,et al.  Intelligent Tutoring Systems , 2000, Lecture Notes in Computer Science.

[17]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[18]  Zachary A. Pardos,et al.  Modelling Student Behavior using Granular Large Scale Action Data from a MOOC , 2016, ArXiv.

[19]  Frank K. Soong,et al.  TTS synthesis with bidirectional LSTM based recurrent neural networks , 2014, INTERSPEECH.

[20]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[21]  Yoav Goldberg,et al.  A Primer on Neural Network Models for Natural Language Processing , 2015, J. Artif. Intell. Res..

[22]  Alvaro Ortigosa,et al.  Recommendation in Higher Education Using Data Mining Techniques , 2009, EDM.

[23]  Carolyn Penstein Rosé,et al.  “ Turn on , Tune in , Drop out ” : Anticipating student dropouts in Massive Open Online Courses , 2013 .

[24]  Hugues Bersini,et al.  Long and Short-Term Recommendations with Recurrent Neural Networks , 2017, UMAP.

[25]  Lise Getoor,et al.  User Preferences for Hybrid Explanations , 2017, RecSys.

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Jacob Whitehill,et al.  Delving Deeper into MOOC Student Dropout Prediction , 2017, ArXiv.

[28]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .

[29]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[30]  Robin van Meteren Using Content-Based Filtering for Recommendation , 2000 .

[31]  George Karypis,et al.  Domain-Aware Grade Prediction and Top-n Course Recommendation , 2016, RecSys.

[32]  Arkadiusz Paterek,et al.  Improving regularized singular value decomposition for collaborative filtering , 2007 .

[33]  Fabiana Vernero,et al.  SoNARS: A Social Networks-Based Algorithm for Social Recommender Systems , 2009, UMAP.

[34]  Patrick Gallinari,et al.  Latent Trajectory Modeling: A Light and Efficient Way to Introduce Time in Recommender Systems , 2015, RecSys.

[35]  Wei Xu,et al.  Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN) , 2014, ICLR.

[36]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[37]  Laurens van der Maaten,et al.  Accelerating t-SNE using tree-based algorithms , 2014, J. Mach. Learn. Res..

[38]  Judy Kay,et al.  Bringing Scrutability to Adaptive Hypertext Teaching , 2000, Intelligent Tutoring Systems.

[39]  Tsvi Kuflik,et al.  Cross-Domain Mediation in Collaborative Filtering , 2007, User Modeling.

[40]  Zachary A. Pardos,et al.  Imputing KCs with Representations of Problem Content and Context , 2017, UMAP.

[41]  Pallab Paul,et al.  A comparison of concurrent and retrospective verbal protocol analysis. , 2000, The American journal of psychology.

[42]  Sean M. McNee,et al.  Getting to know you: learning new user preferences in recommender systems , 2002, IUI '02.

[43]  Judy Kay,et al.  Open Learner Models , 2010, Advances in Intelligent Tutoring Systems.

[44]  Hugh C. Davis,et al.  Evaluating Three Scrutability and Three Privacy User Privileges for a Scrutable User Modelling Infrastructure , 2009, UMAP.

[45]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[46]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[47]  Omer Levy,et al.  Dependency-Based Word Embeddings , 2014, ACL.

[48]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[49]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[50]  Yong Liu,et al.  Improved Recurrent Neural Networks for Session-based Recommendations , 2016, DLRS@RecSys.

[51]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[52]  Dan Frankowski,et al.  Collaborative Filtering Recommender Systems , 2007, The Adaptive Web.

[53]  Dietmar Jannach,et al.  User Modeling and User-Adapted Interaction Session-based Item Recommendation in E-Commerce On Short-Term Intents , Reminders , Trends , and Discounts , 2017 .

[54]  Judy Kay,et al.  Scrutability as a core interface element , 2005, AIED.

[55]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[56]  Michael J. Pazzani,et al.  A Framework for Collaborative, Content-Based and Demographic Filtering , 1999, Artificial Intelligence Review.

[57]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[58]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[59]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[60]  Zachary A. Pardos,et al.  Enabling Real-Time Adaptivity in MOOCs with a Personalized Next-Step Recommendation Framework , 2017, L@S.

[61]  A. Diamond,et al.  Interventions Shown to Aid Executive Function Development in Children 4 to 12 Years Old , 2011, Science.

[62]  David M. Blei,et al.  Modeling User Exposure in Recommendation , 2015, WWW.

[63]  Zhen Li,et al.  Visual planner: beyond prerequisites, designing an interactive course planner for a 21st century flexible curriculum , 2012, CHI EA '12.

[64]  Razvan Pascanu,et al.  Theano: A CPU and GPU Math Compiler in Python , 2010, SciPy.

[65]  Vincent Aleven,et al.  Instruction Based on Adaptive Learning Technologies , 2016 .

[66]  Zachary A. Pardos,et al.  Analysis of Student Behaviour in Habitable Worlds Using Continuous Representation Visualization , 2017, ArXiv.

[67]  Peter Brusilovsky,et al.  Encouraging user participation in a course recommender system: An impact on user behavior , 2011, Comput. Hum. Behav..

[68]  Xingshe Zhou,et al.  TV Program Recommendation for Multiple Viewers Based on user Profile Merging , 2006, User Modeling and User-Adapted Interaction.

[69]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[70]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[71]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[72]  Aditya G. Parameswaran,et al.  Recommendation systems with complex constraints: A course recommendation perspective , 2011, TOIS.

[73]  Qi Gao,et al.  Analyzing user modeling on twitter for personalized news recommendations , 2011, UMAP'11.

[74]  Barry Smyth,et al.  Using twitter to recommend real-time topical news , 2009, RecSys '09.

[75]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[76]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[77]  Laura K. Allen,et al.  Does agency matter?: Exploring the impact of controlled behaviors within a game-based environment , 2015, Comput. Educ..

[78]  Daniel Thalmann,et al.  A simple but effective method to incorporate trusted neighbors in recommender systems , 2012, UMAP.

[79]  Alex Graves,et al.  DRAW: A Recurrent Neural Network For Image Generation , 2015, ICML.

[80]  Pasquale Lops,et al.  Learning Word Embeddings from Wikipedia for Content-Based Recommender Systems , 2016, ECIR.

[81]  Hung-Hsuan Chen,et al.  Behavior2Vec: Generating Distributed Representations of Users' Behaviors on Products for Recommender Systems , 2018, ACM Trans. Knowl. Discov. Data.

[82]  Peter Brusilovsky,et al.  Methods and techniques of adaptive hypermedia , 1996, User Modeling and User-Adapted Interaction.

[83]  Jenna Burrell,et al.  How the machine ‘thinks’: Understanding opacity in machine learning algorithms , 2016 .

[84]  Li Chen,et al.  A user-centric evaluation framework for recommender systems , 2011, RecSys '11.

[85]  Michael J. Pazzani,et al.  Content-Based Recommendation Systems , 2007, The Adaptive Web.

[86]  Pasquale Lops,et al.  A Deep Architecture for Content-based Recommendations Exploiting Recurrent Neural Networks , 2017, UMAP.

[87]  Razvan Pascanu,et al.  Theano: new features and speed improvements , 2012, ArXiv.

[88]  Bamshad Mobasher,et al.  Recommender Systems as Multistakeholder Environments , 2017, UMAP.

[89]  Jonathan Gemmell,et al.  Infusing Collaborative Recommenders with Distributed Representations , 2016, DLRS@RecSys.

[90]  Tat-Seng Chua,et al.  TEM: Tree-enhanced Embedding Model for Explainable Recommendation , 2018, WWW.

[91]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[92]  Yehuda Koren,et al.  Factor in the neighbors: Scalable and accurate collaborative filtering , 2010, TKDD.

[93]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[94]  Geoffrey E. Hinton,et al.  Learning distributed representations of concepts. , 1989 .

[95]  John Riedl,et al.  Recommender systems: from algorithms to user experience , 2012, User Modeling and User-Adapted Interaction.

[96]  Dragan Gasevic,et al.  Open Learning Analytics: an integrated modularized platform , 2011 .

[97]  Param Vir Singh,et al.  A Hidden Markov Model for Collaborative Filtering , 2010, MIS Q..

[98]  Zachary A. Pardos,et al.  The School of Information and its relationship to computer science at UC Berkeley , 2017 .