CourseQ: the impact of visual and interactive course recommendation in university environments

The abundance of courses available in a university often overwhelms students as they must select courses that are relevant to their academic interests and satisfy their requirements. A large number of existing studies in course recommendation systems focus on the accuracy of prediction to show students the most relevant courses with little consideration on interactivity and user perception. However, recent work has highlighted the importance of user-perceived aspects of recommendation systems, such as transparency, controllability, and user satisfaction. This paper introduces CourseQ, an interactive course recommendation system that allows students to explore courses by using a novel visual interface so as to improve transparency and user satisfaction of course recommendations. We describe the design concepts, interactions, and algorithm of the proposed system. A within-subject user study (N=32) was conducted to evaluate our system compared to a baseline interface without the proposed interactive visualization. The evaluation results show that our system improves many user-centric metrics including user acceptance and understanding of the recommendation results. Furthermore, our analysis of user interaction behaviors in the system indicates that CourseQ could help different users with their course-seeking tasks. Our results and discussions highlight the impact of visual and interactive features in course recommendation systems and inform the design of future recommendation systems for higher education.

[1]  Tobias Höllerer,et al.  LinkedVis: exploring social and semantic career recommendations , 2013, IUI '13.

[2]  Adi Botea,et al.  Where can my career take me?: harnessing dialogue for interactive career goal recommendations , 2019, IUI.

[3]  George Karypis,et al.  Collaborative multi-regression models for predicting students' performance in course activities , 2015, LAK.

[4]  Barry Smyth,et al.  PeerChooser: visual interactive recommendation , 2008, CHI.

[5]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[6]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[7]  Jie Tang,et al.  Guess you like: course recommendation in MOOCs , 2017, WI.

[8]  Peter Brusilovsky,et al.  User-controllable personalization: A case study with SetFusion , 2015, Int. J. Hum. Comput. Stud..

[9]  Huzefa Rangwala,et al.  Course-Specific Markovian Models for Grade Prediction , 2018, PAKDD.

[10]  Zachary A. Pardos,et al.  Combating the Filter Bubble: Designing for Serendipity in a University Course Recommendation System , 2019, ArXiv.

[11]  Robert van Liere,et al.  Overview of interactive visualization , 2009 .

[12]  Harald Steck,et al.  Interactive Recommender Systems: Tutorial , 2015, RecSys.

[13]  S. Simon,et al.  Attitudes towards science: A review of the literature and its implications , 2003 .

[14]  D.H. Lee,et al.  Fighting Information Overflow with Personalized Comprehensive Information Access: A Proactive Job Recommender , 2007, Third International Conference on Autonomic and Autonomous Systems (ICAS'07).

[15]  Björn Hartmann,et al.  The diversity donut: enabling participant control over the diversity of recommended responses , 2011, CHI EA '11.

[16]  Tobias Höllerer,et al.  TasteWeights: a visual interactive hybrid recommender system , 2012, RecSys.

[17]  George Karypis,et al.  Scholars Walk: A Markov Chain Framework for Course Recommendation , 2019, EDM.

[18]  Narimel Bendakir,et al.  Using Association Rules for Course Recommendation , 2006 .

[19]  Li Chen,et al.  A user-centric evaluation framework for recommender systems , 2011, RecSys '11.

[20]  Judith Masthoff,et al.  Designing and Evaluating Explanations for Recommender Systems , 2011, Recommender Systems Handbook.

[21]  Jürgen Ziegler,et al.  Blended Recommending: Integrating Interactive Information Filtering and Algorithmic Recommender Techniques , 2015, CHI.

[22]  Christoph Trattner,et al.  See what you want to see: visual user-driven approach for hybrid recommendation , 2014, IUI.

[23]  Aditya Johri,et al.  Next-Term Student Performance Prediction: A Recommender Systems Approach , 2016, EDM.

[24]  A. M. Madni,et al.  Recommender systems in e-commerce , 2014, 2014 World Automation Congress (WAC).

[25]  Bart P. Knijnenburg,et al.  Each to his own: how different users call for different interaction methods in recommender systems , 2011, RecSys '11.

[26]  Robin D. Burke,et al.  Evaluating the dynamic properties of recommendation algorithms , 2010, RecSys '10.

[27]  Rashmi R. Sinha,et al.  The role of transparency in recommender systems , 2002, CHI Extended Abstracts.

[28]  Hassan Khosravi,et al.  Reciprocal peer recommendation for learning purposes , 2018, LAK.

[29]  Raphaël Morsomme,et al.  Content-based Course Recommender System for Liberal Arts Education , 2019, EDM.

[30]  Zachary A. Pardos,et al.  Goal-based Course Recommendation , 2018, LAK.

[31]  Li Chen,et al.  Evaluating recommender systems from the user’s perspective: survey of the state of the art , 2012, User Modeling and User-Adapted Interaction.

[32]  Zachary A. Pardos,et al.  Connectionist recommendation in the wild: on the utility and scrutability of neural networks for personalized course guidance , 2018, User Modeling and User-Adapted Interaction.

[33]  George Karypis,et al.  Will this Course Increase or Decrease Your GPA? Towards Grade-aware Course Recommendation , 2019, ArXiv.

[34]  Peter Brusilovsky,et al.  Beyond the Ranked List: User-Driven Exploration and Diversification of Social Recommendation , 2018, IUI.

[35]  Izak Benbasat,et al.  E-Commerce Product Recommendation Agents: Use, Characteristics, and Impact , 2007, MIS Q..

[36]  Ed H. Chi.  Transient User Profiling , 2004 .

[37]  Pierre-Majorique Léger,et al.  Working with a Recommendation Agent: How Recommendation Presentation Influences Users' Perceptions and Behaviors , 2018, CHI Extended Abstracts.

[38]  Ben Shneiderman,et al.  Finding Similar People to Guide Life Choices: Challenge, Design, and Evaluation , 2017, CHI.

[39]  Peter Brusilovsky,et al.  A Study on User-Controllable Social Exploratory Search , 2018, IUI.

[40]  Kirsten Swearingen,et al.  Beyond Algorithms: An HCI Perspective on Recommender Systems , 2001 .

[41]  John Riedl,et al.  Recommender systems: from algorithms to user experience , 2012, User Modeling and User-Adapted Interaction.

[42]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[43]  Andreas Kerren,et al.  Analysis of VINCI 2009-2017 Proceedings , 2018, VINCI.

[44]  Vincent Tinto,et al.  Classrooms as Communities: Exploring the Educational Character of Student Persistence. , 1997 .

[45]  Philip S. Yu,et al.  A Score Prediction Approach for Optional Course Recommendation via Cross-User-Domain Collaborative Filtering , 2019, IEEE Access.

[46]  Lior Rokach,et al.  Recommender Systems Handbook , 2010 .

[47]  S. B. Aher,et al.  Combination of machine learning algorithms for recommendation of courses in E-Learning System based on historical data , 2013, Knowl. Based Syst..

[48]  George Karypis,et al.  Domain-Aware Grade Prediction and Top-n Course Recommendation , 2016, RecSys.

[49]  Min Lu,et al.  Exploration and Explanation: An Interactive Course Recommendation System for University Environments , 2021, IUI Workshops.

[50]  Sean M. McNee,et al.  Making recommendations better: an analytic model for human-recommender interaction , 2006, CHI Extended Abstracts.

[51]  Katrien Verbert,et al.  Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities , 2016, Expert Syst. Appl..

[52]  Nyi Nyi Htun,et al.  Explaining and exploring job recommendations: a user-driven approach for interacting with knowledge-based job recommender systems , 2019, RecSys.

[53]  Tobias Höllerer,et al.  SmallWorlds: Visualizing Social Recommendations , 2010, Comput. Graph. Forum.

[54]  Jürgen Ziegler,et al.  Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems , 2019, CHI.

[55]  Denis Parra,et al.  Moodplay: Interactive Mood-based Music Discovery and Recommendation , 2016, UMAP.

[56]  John O'Donovan,et al.  Moodplay: Interactive music recommendation based on Artists' mood similarity , 2019, Int. J. Hum. Comput. Stud..

[57]  Katrien Verbert,et al.  How Do Different Levels of User Control Affect Cognitive Load and Acceptance of Recommendations? , 2017, IntRS@RecSys.

[58]  Alfred Kobsa,et al.  Inspectability and control in social recommenders , 2012, RecSys.

[59]  Andreas Kerren,et al.  A survey of surveys on the use of visualization for interpreting machine learning models , 2020, Inf. Vis..

[60]  Ben Shneiderman,et al.  EventAction , 2019, ACM Trans. Interact. Intell. Syst..

[61]  B. Shneiderman,et al.  EventAction , 2019 .