An Evaluation Methodology for Collaborative Recommender Systems

Recommender systems use statistical and knowledge discovery techniques in order to recommend products to users and to mitigate the problem of information overload. The evaluation of the quality of recommender systems has become an important issue for choosing the best learning algorithms. In this paper we propose an evaluation methodology for collaborative filtering (CF) algorithms. This methodology carries out a clear, guided and repeatable evaluation of a CF algorithm. We apply the methodology on two datasets, with different characteristics, using two CF algorithms: singular value decomposition and naive bayesian networks.

[1]  J Allan,et al.  Readings in information retrieval. , 1998 .

[2]  John Riedl,et al.  An algorithmic framework for performing collaborative filtering , 1999, SIGIR '99.

[3]  John Zimmerman,et al.  Personalization: Improving Ease-of-Use, Trust and Accuracy of a TV Show Recommender , 2002 .

[4]  David G. Stork,et al.  Pattern Classification , 1973 .

[5]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[6]  John Riedl,et al.  Analysis of recommendation algorithms for e-commerce , 2000, EC '00.

[7]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[8]  Pattie Maes,et al.  Social information filtering: algorithms for automating “word of mouth” , 1995, CHI '95.

[9]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[10]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[11]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[12]  James Bennett,et al.  The Netflix Prize , 2007 .

[13]  Michael J. Pazzani,et al.  Learning Collaborative Information Filters , 1998, ICML.

[14]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[15]  John Riedl,et al.  Application of Dimensionality Reduction in Recommender Systems , 2000 .

[16]  William W. Cohen,et al.  Recommendation as Classification: Using Social and Content-Based Information in Recommendation , 1998, AAAI/IAAI.

[17]  John Riedl,et al.  Application of Dimensionality Reduction in Recommender System - A Case Study , 2000 .

[18]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[19]  Dimitris Plexousakis,et al.  Qualitative analysis of user-based and item-based prediction algorithms for recommendation agents , 2004, Eng. Appl. Artif. Intell..

[20]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.