A Theoretical Analysis of Two-Stage Recommendation for Cold-Start Collaborative Filtering

In this paper, we present a theoretical framework for tackling the cold-start collaborative filtering problem, where unknown targets (items or users) keep coming to the system, and there is a limited number of resources (users or items) that can be allocated and related to them. The solution requires a trade-off between exploitation and exploration since with the limited recommendation opportunities, we need to, on one hand, allocate the most relevant resources right away, but, on the other hand, it is also necessary to allocate resources that are useful for learning the target's properties in order to recommend more relevant ones in the future. In this paper, we study a simple two-stage recommendation combining a sequential and a batch solution together. We first model the problem with the partially observable Markov decision process (POMDP) and provide its exact solution. Then, through an in-depth analysis over the POMDP value iteration solution, we identify that an exact solution can be abstracted as selecting resources that are not only highly relevant to the target according to the initial-stage information, but also highly correlated, either positively or negatively, with other potential resources for the next stage. With this finding, we propose an approximate solution to ease the intractability of the exact solution. Our initial results on synthetic data and the MovieLens 100K dataset confirm our theoretical development and analysis.

[1]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[2]  Weinan Zhang,et al.  Optimal real-time bidding for display advertising , 2014, KDD.

[3]  Andreas Krause,et al.  Explore-exploit in top-N recommender systems via Gaussian processes , 2014, RecSys '14.

[4]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[5]  Jun Wang,et al.  Unified relevance models for rating prediction in collaborative filtering , 2008, TOIS.

[6]  Oren Somekh,et al.  Budget-Constrained Item Cold-Start Handling in Collaborative Filtering Recommenders via Optimal Design , 2014, WWW.

[7]  Nicholas J. Belkin,et al.  Information filtering and information retrieval: two sides of the same coin? , 1992, CACM.

[8]  ChengXiang Zhai,et al.  Implicit user modeling for personalized search , 2005, CIKM '05.

[9]  John Riedl,et al.  Combining Collaborative Filtering with Personal Agents for Better Recommendations , 1999, AAAI/IAAI.

[10]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[11]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[12]  Masashi Sugiyama,et al.  Influence-based collaborative active learning , 2007, RecSys '07.

[13]  Kevin D. Glazebrook,et al.  Multi-Armed Bandit Allocation Indices: Gittins/Multi-Armed Bandit Allocation Indices , 2011 .

[14]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[15]  Rubens Neil,et al.  Output divergence criterion for active learning in collaborative settings (数理モデル化と問題解決・バイオ情報学) , 2008 .

[16]  Frederi G. Viens,et al.  Stein's lemma, Malliavin calculus, and tail bounds, with application to polymer fluctuation exponent , 2009, 0901.0383.

[17]  Thomas Hofmann,et al.  Latent semantic models for collaborative filtering , 2004, TOIS.

[18]  Thomas J. Walsh,et al.  Exploring compact reinforcement-learning representations with linear regression , 2009, UAI.

[19]  Yiming Yang,et al.  Personalized active learning for collaborative filtering , 2008, SIGIR '08.

[20]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[21]  Jun Wang,et al.  Interactive exploratory search for multi page search results , 2013, WWW.

[22]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[23]  Yehuda Koren,et al.  Improved Neighborhood-based Collaborative Filtering , 2007 .

[24]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[25]  John Riedl,et al.  An algorithmic framework for performing collaborative filtering , 1999, SIGIR '99.

[26]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[27]  David M. Pennock,et al.  Categories and Subject Descriptors , 2001 .

[28]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[29]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[30]  John Riedl,et al.  An Empirical Analysis of Design Choices in Neighborhood-Based Collaborative Filtering Algorithms , 2002, Information Retrieval.

[31]  Jun Wang,et al.  Adaptive diversification of recommendation results via latent factor portfolio , 2012, SIGIR '12.

[32]  J. Bather,et al.  Multi‐Armed Bandit Allocation Indices , 1990 .

[33]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[34]  S. Robertson.  The probability ranking principle in IR , 1997 .

[35]  Christopher Meek,et al.  Tied boltzmann machines for cold start recommendations , 2008, RecSys '08.

[36]  John Riedl,et al.  Learning preferences of new users in recommender systems: an information theoretic approach , 2008, SKDD.

[37]  Shuang-Hong Yang,et al.  Functional matrix factorizations for cold-start recommendation , 2011, SIGIR.

[38]  Rasoul Karimi,et al.  Active Learning for Recommender Systems , 2015, KI - Künstliche Intelligenz.

[39]  Christian M. Ernst,et al.  Multi-armed Bandit Allocation Indices , 1989 .

[40]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[41]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[42]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[43]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[44]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[45]  Stephen E. Robertson,et al.  Probabilistic relevance ranking for collaborative filtering , 2008, Information Retrieval.

[46]  Jun Wang,et al.  Interactive collaborative filtering , 2013, CIKM.

[47]  田口 玄一,et al.  Introduction to quality engineering : designing quality into products and processes , 1986 .

[48]  Yehuda Koren,et al.  Adaptive bootstrapping of recommender systems using decision trees , 2011, WSDM '11.

[49]  Sean M. McNee,et al.  Getting to know you: learning new user preferences in recommender systems , 2002, IUI '02.

[50]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..