Social Influence Bias in Recommender Systems : A Methodology for Learning , Analyzing , and Mitigating Bias in Ratings

To facilitate browsing and selection, almost all recommender systems display an aggregate statistic (the average/mean or median rating value) for each item. This value has potential to influence a participant’s individual rating for an item due to what is known in the survey and psychology literature as Social Influence Bias; the tendency for individuals to conform to what they perceive as the “norm” in a community. As a result, ratings can be closer to the average and less diverse than they would be otherwise. We propose a methodology to 1) learn, 2) analyze, and 3) mitigate the effect of social influence bias in recommender systems. In the Learning phase, a baseline dataset is established with an initial set of participants by allowing them to rate items twice: before seeing the median rating, and again after seeing it. In the Analysis phase, a new non-parametric significance test based on the Wilcoxon statistic can quantify the extent of social influence bias in this data. If this bias is significant, we propose a Mitigation phase where mathematical models are constructed from this data using polynomial regression and the Bayesian Information Criterion (BIC) and then inverted to produce a filter that can reduce the effect of social influence bias. As a case study, we apply this methodology to the California Report Card (CRC), a new recommender system that encourages political engagement. After the Learning phase collected 9390 ratings, the non-parametric test in the Analysis phase rejected the null hypothesis, identifying significant social influence bias: ratings after display of the median were on average 19.3% closer to the median value. In the Mitigating phase, the learned polynomial models were able to predict changed ratings with a normalized RMSE of 12.8% and reduce bias by 76.3%. Results suggest that social influence bias can be significant in recommender systems and that this bias can be substantially reduced with machine learning. The CRC, our data, and experimental code can be found at: http://californiareportcard.org/data/