Orienting Students to Course Recommendations Using Three Types of Explanation

An emerging challenge in course recommender systems is explaining to students why they have been recommended particular courses. In the context of a university, it can be valuable for a recommender system to introduce students to courses they may have not otherwise taken but which are still relevant to them. However, there is a tension between these goals as students have less ability to judge the relevance of courses, the less familiar they are with them. In this paper, we explore ways of familiarizing students with recommendations with three types of explanations designed with varying levels of personalization. We conduct a 67 student randomized controlled experiment using two course recommendation engines, content and context-based, and augment recommendations with three different types of explanation. Students rated each course recommended in terms of novelty, unexpectedness, and successfulness (i.e., intent to enroll). We find several statistically significant results, including an increase in serendipity (i.e., unexpectedness + successfulness) when explaining new course recommendations using keywords from courses a student has previously taken.

[1]  Peter Brusilovsky,et al.  Social Navigation Support in a Course Recommendation System , 2006, AH.

[2]  Nava Tintarev.  The Effectiveness of Personalized Movie Explanations: An Experiment Using Commercial Meta-data , 2008, AH.

[3]  Dorota Glowacka,et al.  Directing exploratory search: reinforcement learning from user interactions with keywords , 2013, IUI '13.

[4]  Annika Wærn,et al.  User Involvement in Automatic Filtering: An Experimental Study , 2004, User Modeling and User-Adapted Interaction.

[5]  Zachary A. Pardos,et al.  Goal-based Course Recommendation , 2018, LAK.

[6]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[7]  Martin Dillon,et al.  Introduction to modern information retrieval: G. Salton and M. McGill. McGraw-Hill, New York (1983). xv + 448 pp., $32.95 ISBN 0-07-054484-0 , 1983 .

[8]  Dietmar Jannach,et al.  User Modeling and User-Adapted Interaction Session-based Item Recommendation in E-Commerce On Short-Term Intents , Reminders , Trends , and Discounts , 2017 .

[9]  Zachary A. Pardos,et al.  Design and Deployment of a Better Course Search Tool: Inferring Latent Keywords from Enrollment Networks , 2019, EC-TEL.

[10]  Peter Brusilovsky,et al.  Open user profiles for adaptive news systems: help or harm? , 2007, WWW '07.

[11]  Zhen Li,et al.  Visual planner: beyond prerequisites, designing an interactive course planner for a 21st century flexible curriculum , 2012, CHI EA '12.

[12]  Jun Wang,et al.  Explanation Mining: Post Hoc Interpretability of Latent Factor Models for Recommendation Systems , 2018, KDD.

[13]  Judith Masthoff,et al.  A Survey of Explanations in Recommender Systems , 2007, 2007 IEEE 23rd International Conference on Data Engineering Workshop.

[14]  George Karypis,et al.  Domain-Aware Grade Prediction and Top-n Course Recommendation , 2016, RecSys.

[15]  Birgitta König-Ries,et al.  An approach to controlling user models and personalization effects in recommender systems , 2013, IUI '13.

[16]  Zachary A. Pardos,et al.  Combating the Filter Bubble: Designing for Serendipity in a University Course Recommendation System , 2019, ArXiv.

[17]  Zachary A. Pardos,et al.  Data-Assistive Course-to-Course Articulation Using Machine Translation , 2019, L@S.

[18]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[19]  Guy Shani,et al.  Evaluating Recommendation Systems , 2011, Recommender Systems Handbook.

[20]  Ruihui Mu,et al.  A Survey of Recommender Systems Based on Deep Learning , 2018, IEEE Access.

[21]  Aditya G. Parameswaran,et al.  Recommendation systems with complex constraints: A course recommendation perspective , 2011, TOIS.

[22]  Ramesh Johari,et al.  How a data-driven course planning tool affects college students' GPA: evidence from two field experiments , 2018, L@S.

[23]  Chris Wong,et al.  Sequence Based Course Recommender for Personalized Curriculum Planning , 2018, AIED.

[24]  Jiawei Han,et al.  Phrase Mining from Massive Text and Its Applications , 2017, Synthesis Lectures on Data Mining and Knowledge Discovery.

[25]  Mouzhi Ge,et al.  How should I explain? A comparison of different explanation types for recommender systems , 2014, Int. J. Hum. Comput. Stud..

[26]  George Karypis,et al.  Scholars Walk: A Markov Chain Framework for Course Recommendation , 2019, EDM.

[27]  Narimel Bendakir,et al.  Using Association Rules for Course Recommendation , 2006 .

[28]  Amelia Zafra,et al.  A Hybrid Multi-Criteria approach using a Genetic Algorithm for Recommending Courses to University Students , 2018, EDM.

[29]  Z. Pardos,et al.  A university map of course knowledge , 2020, PloS one.

[30]  Judith Masthoff,et al.  Explaining Recommendations: Design and Evaluation , 2015, Recommender Systems Handbook.

[31]  Dietmar Jannach,et al.  A systematic review and taxonomy of explanations in decision support and recommender systems , 2017, User Modeling and User-Adapted Interaction.

[32]  Raymond J. Mooney,et al.  Explaining Recommendations: Satisfaction vs. Promotion , 2005 .

[33]  Xu Chen,et al.  Explainable Recommendation: A Survey and New Perspectives , 2018, Found. Trends Inf. Retr..

[34]  John Riedl,et al.  Tagsplanations: explaining recommendations using tags , 2009, IUI.

[35]  Tomoko Ohkuma,et al.  Explaining Recommendations Using Contexts , 2018, IUI.

[36]  Guokun Lai,et al.  Explicit factor models for explainable recommendation based on phrase-level sentiment analysis , 2014, SIGIR.

[37]  Stephanie D. Teasley,et al.  Conceptualizing co-enrollment: accounting for student experiences across the curriculum , 2018, LAK.

[38]  Alfred Kobsa,et al.  Inspectability and control in social recommenders , 2012, RecSys.

[39]  Li Chen,et al.  Trust-inspiring explanation interfaces for recommender systems , 2007, Knowl. Based Syst..

[40]  Zachary A. Pardos,et al.  Connectionist recommendation in the wild: on the utility and scrutability of neural networks for personalized course guidance , 2018, User Modeling and User-Adapted Interaction.

[41]  Kurt Driessens,et al.  Discovering Hidden Course Requirements and Student Competences from Grade Data , 2017, UMAP.

[42]  Jianmo Ni,et al.  Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects , 2019, EMNLP.