Letting Users Choose Recommender Algorithms: An Experimental Study

Recommender systems are not one-size-fits-all; different algorithms and data sources have different strengths, making them a better or worse fit for different users and use cases. As one way of taking advantage of the relative merits of different algorithms, we gave users the ability to change the algorithm providing their movie recommendations and studied how they make use of this power. We conducted our study with the launch of a new version of the MovieLens movie recommender that supports multiple recommender algorithms and allows users to choose the algorithm they want to provide their recommendations. We examine log data from user interactions with this new feature to under-stand whether and how users switch among recommender algorithms, and select a final algorithm to use. We also look at the properties of the algorithms as they were experienced by users and examine their relationships to user behavior. We found that a substantial portion of our user base (25%) used the recommender-switching feature. The majority of users who used the control only switched algorithms a few times, trying a few out and settling down on an algorithm that they would leave alone. The largest number of users prefer a matrix factorization algorithm, followed closely by item-item collaborative filtering; users selected both of these algorithms much more often than they chose a non-personalized mean recommender. The algorithms did produce measurably different recommender lists for the users in the study, but these differences were not directly predictive of user choice.

[1]  Elaine Rich,et al.  User Modeling via Stereotypes , 1998, Cogn. Sci..

[2]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[3]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[4]  Yoav Shoham,et al.  Fab: content-based, collaborative recommendation , 1997, CACM.

[5]  Nicholas J. Belkin,et al.  Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback , 2001, Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.

[6]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[7]  Judy Kay,et al.  The justified user model: A viewable explained user model , 2003 .

[8]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[9]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[10]  Sean M. McNee,et al.  Don't look stupid: avoiding pitfalls when recommending research papers , 2006, CSCW '06.

[11]  Judy Kay,et al.  Scrutable Adaptation: Because We Can and Must , 2006, AH.

[12]  Nava Tintarev,et al.  Explanations of recommendations , 2007, RecSys '07.

[13]  Arkadiusz Paterek,et al.  Improving regularized singular value decomposition for collaborative filtering , 2007 .

[14]  Ron Kohavi,et al.  Responsible editor: R. Bayardo. , 2022 .

[15]  Joseph Sill,et al.  Feature-Weighted Linear Stacking , 2009, ArXiv.

[16]  John Riedl,et al.  Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit , 2011, RecSys '11.

[17]  Bart P. Knijnenburg,et al.  Explaining the user experience of recommender systems , 2012, User Modeling and User-Adapted Interaction.

[18]  John Riedl,et al.  The Tag Genome: Encoding Community Knowledge to Support Novel Interaction , 2012, TIIS.

[19]  John Riedl,et al.  When recommenders fail: predicting recommender failure for algorithm selection and combination , 2012, RecSys.

[20]  Christophe Diot,et al.  Finding a needle in a haystack of reviews: cold start context-based hotel recommender system , 2012, RecSys.

[21]  Eli Pariser,et al.  The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think , 2012 .

[22]  Franca Garzotto,et al.  User effort vs. accuracy in rating-based elicitation , 2012, RecSys.

[23]  Declan Fleming.  Try this at home , 2013 .

[24]  Simon Dooms,et al.  Dynamic generation of personalized hybrid recommender systems , 2013, RecSys.

[25]  Sahin Albayrak,et al.  User-centric evaluation of a K-furthest neighbor collaborative filtering recommender algorithm , 2013, CSCW.

[26]  F. Maxwell Harper,et al.  User perception of differences in recommender algorithms , 2014, RecSys '14.

[27]  Loren G. Terveen,et al.  Exploring the filter bubble: the effect of using recommender systems on content diversity , 2014, WWW.

[28]  Jeffrey T. Hancock,et al.  Experimental evidence of massive-scale emotional contagion through social networks , 2014, Proceedings of the National Academy of Sciences.

[29]  Aaron Halfaker,et al.  User Session Identification Based on Strong Regularities in Inter-activity Time , 2014, WWW.

[30]  F. M. Harper,et al.  Using Groups of Items for Preference Elicitation in Recommender Systems , 2015, CSCW 2015.