Exploiting Performance Estimates for Augmenting Recommendation Ensembles

Ensembling multiple recommender systems via stacking has shown to be effective at improving collaborative recommendation. Recent work extends stacking to use additional user performance predictors (e.g., the total number of ratings made by the user) to help determine how much each base recommender should contribute to the ensemble. Nonetheless, despite the cost of handcrafting discriminative predictors, which typically requires deep knowledge of the strengths and weaknesses of each recommender in the ensemble, only minor improvements have been observed. To overcome this limitation, instead of engineering complex features to predict the performance of different recommenders for a given user, we propose to directly estimate these performances by leveraging the user’s own historical ratings. Experiments on real-world datasets from multiple domains demonstrate that using performance estimates as additional features can significantly improve the accuracy of state-of-the-art ensemblers, achieving nDCG@20 improvements by an average of 23% over not using them.

[1]  Javed A. Aslam,et al.  Models for metasearch , 2001, SIGIR '01.

[2]  L. Breiman.  Stacked Regressions , 1996, Machine Learning.

[3]  W. Bruce Croft,et al.  Predicting query performance , 2002, SIGIR '02.

[4]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[5]  Oren Kurland,et al.  Query-performance prediction: setting the expectations straight , 2014, SIGIR.

[6]  Yehuda Koren,et al.  Factor in the neighbors: Scalable and accurate collaborative filtering , 2010, TKDD.

[7]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[8]  Sophie Ahrens,et al.  Recommender Systems , 2012 .

[9]  Joseph Sill,et al.  Feature-Weighted Linear Stacking , 2009, ArXiv.

[10]  Yehuda Koren,et al.  The BellKor Solution to the Netflix Grand Prize , 2009 .

[11]  Rodrygo L. T. Santos,et al.  Efficient Bayesian Methods for Graph-based Recommendation , 2016, RecSys.

[12]  Marcos André Gonçalves,et al.  A Multicriteria Evaluation of Hybrid Recommender Systems: On the Usefulness of Input Data Characteristics , 2017, ICEIS.

[13]  Josephine Griffith,et al.  Investigations into user rating information and predictive accuracy in a collaborative filtering domain , 2012, SAC '12.

[14]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[15]  Srujana Merugu,et al.  A scalable collaborative filtering framework based on co-clustering , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[16]  Elad Yom-Tov,et al.  Estimating the query difficulty for information retrieval , 2010, Synthesis Lectures on Information Concepts, Retrieval, and Services.

[17]  Robert A. Legenstein,et al.  Combining predictions for accurate recommender systems , 2010, KDD.

[18]  Xinlong Bao,et al.  Stacking recommendation engines with additional meta-features , 2009, RecSys '09.

[19]  Gilles Louppe,et al.  Understanding variable importances in forests of randomized trees , 2013, NIPS.

[20]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[21]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[22]  Paul Resnick,et al.  Recommender systems , 1997, CACM.

[23]  Guy Shani,et al.  Boosting Simple Collaborative Filtering Models Using Ensemble Methods , 2012, ArXiv.

[24]  Claudia Hauff,et al.  Predicting the effectiveness of queries and retrieval systems , 2010, SIGF.

[25]  Alejandro Bellogín,et al.  Predicting the Performance of Recommender Systems: An Information Theoretic Approach , 2011, ICTIR.

[26]  Leo Breiman,et al.  Stacked regressions , 2004, Machine Learning.

[27]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[28]  Craig MacDonald,et al.  On the usefulness of query features for learning to rank , 2012, CIKM.

[29]  Armelle Brun,et al.  Identifying Users with Atypical Preferences to Anticipate Inaccurate Recommendations , 2015, WEBIST.

[30]  Roberto Turrin,et al.  Performance of recommender algorithms on top-n recommendation tasks , 2010, RecSys '10.

[31]  Javier Parapar,et al.  Combining Top-N Recommenders with Metasearch Algorithms , 2017, SIGIR.

[32]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[33]  Gary Marchionini,et al.  Synthesis Lectures on Information Concepts, Retrieval, and Services , 2009 .

[34]  Charu C. Aggarwal,et al.  Ensemble-Based and Hybrid Recommender Systems , 2016 .

[35]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.