Evolving Communities of Recommenders: A Temporal Evaluation

Collaborative Filtering (CF) is the established algorithm that fuels the success of recommender systems. The assumption that these systems rely on is that like-minded users can successfully share the experiences they have had with the content provided; CF is thus a means of spreading information via similarity. There are many ways of measuring how similar two users are, and the predictive power of each method has traditionally been quantified by measuring the distance between predictions and the actual ratings provided by users. However, the data that is available to CF researchers is temporal in its nature; it grows as users respond to recommendations and rate items. The temporal nature of the data is highly influential in determining how accurate and useful any particular prediction will be in the actual deployment of a recommender system. In this work we perform a full evaluation of CF algorithms that includes time, by only considering the currently available information when making predictions. Unfortunately, traditional methods of measuring performance in this context are no longer informative. To overcome this, we introduce a classification of predictions based on confidence, and show that both the parameters used to tune CF and the methods used to measure user similarity will have a strong effect on the predictive confidence of the