A Probabilistic Model for Diversifying Recommendation Lists

We propose a probabilistic method to diversify the results of collaborative filtering. Recommendation diversity is being studied by many researchers as a critical factor that significantly influences user satisfaction. Unlike conventional approaches to recommendation diversification, we theoretically derived a diversification method. Specifically, our method naturally diversifies a recommendation list by maximizing the probability that a user selects at most one item from the list. For enhanced practicality, we formulate a model for the proposed method on three policies — robust estimation, the use of only purchase history, and the elimination of any hyperparameters controlling the diversity. In this paper, we formally demonstrate that our method is practically superior to conventional diversification methods, and experimentally show that our method is competitive with conventional methods in terms of accuracy and diversity.

[1]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[2]  Lars Schmidt-Thieme,et al.  Taxonomy-driven computation of product recommendations , 2004, CIKM '04.

[3]  David R. Karger,et al.  Less is More Probabilistic Models for Retrieving Fewer Relevant Documents , 2006 .

[4]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[5]  Scott Sanner,et al.  Probabilistic latent maximal marginal relevance , 2010, SIGIR '10.

[6]  Neil J. Hurley,et al.  Novelty and Diversity in Top-N Recommendation -- Analysis and Evaluation , 2011, TOIT.

[7]  T. Minka Estimating a Dirichlet distribution , 2012 .

[8]  Jun Wang,et al.  Workshop on novelty and diversity in recommender systems - DiveRS 2011 , 2011, RecSys '11.

[9]  Yu He,et al.  The YouTube video recommendation system , 2010, RecSys '10.

[10]  Charles L. A. Clarke,et al.  Novelty and diversity in information retrieval evaluation , 2008, SIGIR '08.

[11]  Yiming Yang,et al.  A re-examination of text categorization methods , 1999, SIGIR '99.

[12]  Craig MacDonald,et al.  Selectively diversifying web search results , 2010, CIKM.

[13]  Saul Vargas,et al.  Rank and relevance in novelty and diversity metrics for recommender systems , 2011, RecSys '11.

[14]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Jun Wang,et al.  Adaptive diversification of recommendation results via latent factor portfolio , 2012, SIGIR '12.

[16]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[17]  Craig MacDonald,et al.  Intent-aware search result diversification , 2011, SIGIR.

[18]  Licia Capra,et al.  Temporal diversity in recommender systems , 2010, SIGIR.

[19]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[20]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[21]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[22]  Filip Radlinski,et al.  Improving personalized web search using result diversification , 2006, SIGIR.