Conformal matrix factorization based recommender system

Abstract Recommender systems can be viewed as prediction systems wherein one can predict the rating a user would give to a particular item. Typically, items having the highest predicted ratings will be recommended to the user. Matrix Factorization techniques are the most widely used methods in predicting the missing ratings of a user-item rating matrix due to their efficiency and accuracy in prediction. However, users do not know how certain these predictions are. Hence, it is important to associate a confidence measure to the predictions which tells the users how certain the system is in making the predictions. Confidence is a very useful concept in recommender systems which can help users in making decisions about which movies to watch, which books to buy and so on. Different approaches have been proposed to estimate the confidence in recommender systems but none of them provide a guarantee on the error rate of these predictions. Conformal Prediction is a framework which makes it possible to control the number of erroneous predictions by selecting a suitable confidence level which can be varied depending on the application. In this paper, we propose Conformal Matrix Factorization (CMF) and show different ways of adapting conformal prediction to matrix factorization. Nonconformity measure is a key concept of conformal prediction which needs to be defined by careful investigation of the underlying algorithm. We propose and analyse different nonconformity measures based on matrix factorization and experimentally show that the accuracy of the best nonconformity measure is very close to the accuracy of the underlying algorithm.We show that the proposed method yields promising results in terms of prediction accuracy along with confidence measure.

[1]  Alexander Gammerman,et al.  Pattern Recognition and Density Estimation under the General i.i.d. Assumption , 2001, COLT/EuroCOLT.

[2]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[3]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[4]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[5]  Gerhard Friedrich,et al.  Recommender Systems - An Introduction , 2010 .

[6]  Alexander Gammerman,et al.  Transductive Confidence Machines for Pattern Recognition , 2002, ECML.

[7]  Siddhartha Bhattacharyya,et al.  Confidence in predictions from random tree ensembles , 2011, 2011 IEEE 11th International Conference on Data Mining.

[8]  Jonathan L. Herlocker,et al.  A collaborative filtering algorithm and evaluation metric that accurately model the user experience , 2004, SIGIR '04.

[9]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[10]  Yoav Shoham,et al.  Fab: content-based, collaborative recommendation , 1997, CACM.

[11]  Harris Papadopoulos,et al.  Qualified Prediction for Large Data Sets in the Case of Pattern Recognition , 2002, International Conference on Machine Learning and Applications.

[12]  Yoav Shoham,et al.  Content-Based, Collaborative Recommendation. , 1997 .

[13]  Henrik Boström,et al.  Conformal Prediction Using Decision Trees , 2013, 2013 IEEE 13th International Conference on Data Mining.

[14]  Harris Papadopoulos,et al.  Inductive Confidence Machines for Regression , 2002, ECML.

[15]  Ilia Nouretdinov,et al.  Prediction with Confidence Based on a Random Forest Classifier , 2010, AIAI.

[16]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[17]  Harris Papadopoulos,et al.  Reliable Confidence Intervals for Software Effort Estimation , 2009, AIAI Workshops.

[18]  Mark Rosenstein,et al.  Recommending and evaluating choices in a virtual community of use , 1995, CHI '95.

[19]  Henrik Boström,et al.  Evolved decision trees as conformal predictors , 2013, 2013 IEEE Congress on Evolutionary Computation.

[20]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[21]  Tommi S. Jaakkola,et al.  Maximum-Margin Matrix Factorization , 2004, NIPS.

[22]  Harris Papadopoulos,et al.  Inductive Conformal Prediction: Theory and Application to Neural Networks , 2008 .

[23]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.

[24]  Sean M. McNee,et al.  Confidence Displays and Training in Recommender Systems , 2003, INTERACT.

[25]  Alexander Gammerman,et al.  Qualified predictions for microarray and proteomics pattern diagnostics with confidence machines , 2005, Int. J. Neural Syst..

[26]  Gediminas Adomavicius,et al.  Towards more confident recommendations: Improving recommender systems using filtering approach based on rating variance , 2007 .

[27]  G. Shafer,et al.  Algorithmic Learning in a Random World , 2005 .

[28]  Guy Shani,et al.  Evaluating Recommendation Systems , 2011, Recommender Systems Handbook.

[29]  Harris Papadopoulos,et al.  Regression Conformal Prediction with Nearest Neighbours , 2014, J. Artif. Intell. Res..

[30]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[31]  Harris Papadopoulos,et al.  Confidence Predictions for the Diagnosis of Acute Abdominal Pain , 2009, AIAI.

[32]  Alexander Gammerman,et al.  Criteria of Efficiency for Conformal Prediction , 2016, COPA.

[33]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[34]  Vladimir Vovk,et al.  Ridge Regression Confidence Machine , 2001, International Conference on Machine Learning.

[35]  Maciej A. Mazurowski,et al.  Estimating confidence of individual rating predictions in collaborative filtering recommender systems , 2013, Expert Syst. Appl..

[36]  Harris Papadopoulos,et al.  Conformal Prediction with Neural Networks , 2007 .

[37]  Guy Shani,et al.  A Survey of Accuracy Evaluation Metrics of Recommendation Tasks , 2009, J. Mach. Learn. Res..

[38]  Jin Zhang,et al.  Recognition of hypoxia EEG with a preset confidence level based on EEG analysis , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[39]  Alexander Gammerman,et al.  Computationally Efficient Transductive Machines , 2000, ALT.

[40]  Zhiyuan Luo,et al.  Network Traffic Demand Prediction with Confidence , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[41]  Vladimir Vovk,et al.  Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications , 2014 .

[42]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[43]  Alexander Gammerman,et al.  Serum Proteomic Abnormality Predating Screen Detection of Ovarian Cancer , 2009, Comput. J..

[44]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[45]  Yehuda Koren,et al.  OrdRec: an ordinal model for predicting personalized item rating distributions , 2011, RecSys '11.