Solving the apparent diversity-accuracy dilemma of recommender systems

Recommender systems use data on past user preferences to predict possible future likes and interests. A key challenge is that while the most useful individual recommendations are to be found among diverse niche objects, the most reliably accurate results are obtained by methods that recommend objects based on user or object similarity. In this paper we introduce a new algorithm specifically to address the challenge of diversity and show how it can be used to resolve this apparent dilemma when combined in an elegant hybrid with an accuracy-focused algorithm. By tuning the hybrid appropriately we are able to obtain, without relying on any semantic or context-specific information, simultaneous gains in both accuracy and diversity of recommendations.

[1]  Yi-Cheng Zhang,et al.  Effect of initial configuration on network-based recommendation , 2007, 0711.2506.

[2]  Amrit Tiwana,et al.  Integrating knowledge on the Web , 2001, IEEE Internet Computing.

[3]  Andreas Hotho,et al.  Information Retrieval in Folksonomies: Search and Ranking , 2006, ESWC.

[4]  Yi-Cheng Zhang,et al.  Personalized Recommendation via Integrated Diffusion on User-Item-Tag Tripartite Graphs , 2009, ArXiv.

[5]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[6]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[7]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[8]  Peretz Shoval,et al.  Information Filtering: Overview of Issues, Research and Systems , 2001, User Modeling and User-Adapted Interaction.

[9]  Yi-Cheng Zhang,et al.  Information filtering via self-consistent refinement , 2008, 0802.3748.

[10]  Paul Van Dooren,et al.  Reputation Systems and Optimization , 2008 .

[11]  Aleksandar Stojmirovic,et al.  Information Flow in Interaction Networks , 2011, J. Comput. Biol..

[12]  Gerald J. Kowalski,et al.  Information Retrieval Systems , 1997, The Information Retrieval Series.

[13]  M. Tribus.  Thermostatics and thermodynamics , 1961 .

[14]  Yi-Cheng Zhang,et al.  Heat conduction process on community networks as a recommendation model. , 2007, Physical review letters.

[15]  Mark S. Granovetter.  The Strength of Weak Ties , 1973, American Journal of Sociology.

[16]  John Riedl,et al.  E-Commerce Recommendation Applications , 2004, Data Mining and Knowledge Discovery.

[17]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[18]  Kamal Ali,et al.  TiVo: making show recommendations using a distributed collaborative filtering architecture , 2004, KDD.

[19]  S Maslov,et al.  Extracting hidden information from knowledge networks. , 2001, Physical review letters.

[20]  Vittorio Loreto,et al.  Collaborative Tagging and Semiotic Dynamics , 2006, ArXiv.

[21]  Michael J. Pazzani,et al.  Content-Based Recommendation Systems , 2007, The Adaptive Web.

[22]  Kenneth Y. Goldberg,et al.  Eigentaste: A Constant Time Collaborative Filtering Algorithm , 2001, Information Retrieval.

[23]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[24]  James Bennett,et al.  The Netflix Prize , 2007 .

[25]  Paul Resnick,et al.  Recommender systems , 1997, CACM.

[26]  Vittorio Loreto,et al.  Semiotic dynamics and collaborative tagging , 2006, Proceedings of the National Academy of Sciences.

[27]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[28]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[29]  Yi-Cheng Zhang,et al.  Information filtering via Iterative Refinement , 2006, ArXiv.

[30]  Yi-Cheng Zhang,et al.  Manifesto for the Reputation Society , 2004, First Monday.

[31]  Yehuda Koren,et al.  The BellKor Solution to the Netflix Grand Prize , 2009 .

[32]  Mark Claypool,et al.  Inferring User Interest , 2001, IEEE Internet Comput..

[33]  Nicholas J. Belkin,et al.  Helping people find what they don't know , 2000, CACM.

[34]  Yi-Cheng Zhang,et al.  Bipartite network projection and personal recommendation. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[36]  Thomas Hofmann,et al.  Latent semantic models for collaborative filtering , 2004, TOIS.