A scalable P2P recommender system based on distributed collaborative filtering

Collaborative Filtering (CF) technique has been proved to be one of the most successful techniques in recommender systems in recent years. However, most existing CF based recommender systems worked in a centralized way and suffered from its shortage in scalability as their calculation complexity increased quickly both in time and space when the record in user database increases. In this article, we first propose a distributed CF algorithm called PipeCF together with two novel approaches: significance refinement and unanimous amplification, to further improve the scalability and prediction accuracy. We then show how to implement this algorithm on a Peer-to-Peer (P2P) structure through distributed hash table method, which is the most popular and efficient P2P routing algorithm, to construct a scalable distributed recommender system. The experimental data show that the distributed CF-based recommender system has much better scalability than traditional centralized ones with comparable prediction efficiency and accuracy. q 2004 Elsevier Ltd. All rights reserved.

[1]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[2]  Pattie Maes,et al.  Social information filtering: algorithms for automating “word of mouth” , 1995, CHI '95.

[3]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[4]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[5]  Tomas Olsson,et al.  Bootstrapping and Decentralizing Recommender Systems , 2003 .

[6]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[7]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[8]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[9]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[10]  Amund Tveit,et al.  Peer-to-peer based recommendations for mobile commerce , 2001, WMC '01.

[11]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[12]  John F. Canny,et al.  Collaborative filtering with privacy , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[13]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[14]  John Riedl,et al.  An algorithmic framework for performing collaborative filtering , 1999, SIGIR '99.