Rating support interfaces to improve user experience and recommender accuracy

One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to evaluate the efficacy of the new interfaces via a user experiment. Our results suggest that presenting exemplars during the rating process helps users rate more consistently, and increases the quality of the data.

[1]  Neil J. Hurley,et al.  Detecting noise in recommender system databases , 2006, IUI '06.

[2]  Christopher K. Hsee,et al.  The Evaluability Hypothesis: An Explanation for Preference Reversals between Joint and Separate Evaluations of Alternatives , 1996 .

[3]  John Riedl,et al.  Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit , 2011, RecSys '11.

[4]  Mark Rosenstein,et al.  Recommending and evaluating choices in a virtual community of use , 1995, CHI '95.

[5]  John Riedl,et al.  The Tag Genome: Encoding Community Knowledge to Support Novel Interaction , 2012, TIIS.

[6]  I. Simonson,et al.  Determinants of Customers' Responses to Customized Offers: Conceptual Framework and Research Propositions , 2003 .

[7]  Shilad Sen,et al.  Rating: how difficult is it? , 2011, RecSys '11.

[8]  Eric J. Johnson,et al.  Mindful judgment and decision making. , 2009, Annual review of psychology.

[9]  Sean M. McNee,et al.  Interfaces for Eliciting New User Preferences in Recommender Systems , 2003, User Modeling.

[10]  John Riedl,et al.  Is seeing believing?: how recommender system interfaces affect users' opinions , 2003, CHI '03.

[11]  B. Newell.  Judgment Under Uncertainty , 2013 .

[12]  M. F. Luce,et al.  Constructive Consumer Choice Processes , 1998 .

[13]  John Riedl,et al.  Predicting Users' Preference from Tag Relevance , 2013, UMAP.

[14]  John Riedl,et al.  How many bits per rating? , 2012, RecSys.

[15]  B. Fischhoff,et al.  Value elicitation: Is there anything in there? , 1991 .

[16]  A. Tversky,et al.  Contingent weighting in judgment and choice , 1988 .

[17]  Mark P. Graus,et al.  Remembering the stars?: effect of time on preference retrieval from memory , 2012, RecSys.

[18]  Bart P. Knijnenburg,et al.  Explaining the user experience of recommender systems , 2012, User Modeling and User-Adapted Interaction.

[19]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[20]  Nava Tintarev,et al.  Rate it again: increasing recommendation accuracy by user re-rating , 2009, RecSys '09.

[21]  Shawn P. Curley,et al.  Recommender systems, consumer preferences, and anchoring effects , 2011, RecSys 2011.

[22]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[23]  F. Maxwell Harper,et al.  An Economic Model of User Rating in an Online Recommender System , 2005, User Modeling.

[24]  Tamara Munzner,et al.  The design space of opinion measurement interfaces: exploring recall support for rating and ranking , 2012, CHI.

[25]  P. Slovic.  The Construction of Preference , 1995 .

[26]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[27]  Till Plumbaum,et al.  Users and noise: the magic barrier of recommender systems , 2012, UMAP.

[28]  T. Mussweiler.  Comparison processes in social judgment: mechanisms and consequences. , 2003, Psychological review.