A comprehensive approach for the evaluation of recommender systems using implicit feedback

Evaluation strategies are essentials in assessing the degree of satisfaction that recommender systems can provide to users. The evaluation schemes rely heavily on user feedback, however these feedbacks may be casual, biased or spam which leads to an inappropriate evaluation. In this paper, a comprehensive approach for the evaluation of recommendation system is proposed. The implicit user feedbacks are taken for the different products on the basis of the reviews provided to them. A novel sincerity check mechanism is suggested to render the biasedness and casual among the users. Further, mathematical model is presented to classify the products preference criteria. The list of the preferred products yield different ranking. Rank aggregation algorithm is used to obtain a final ranking, which is compared with the base ranking to be evaluated. Hence, with the help of suggested methodology, an evaluation strategy is suggested that avoids the risk of fake and biased feedbacks. The comparison of the proposed approach with existing schemes shows the superiority of the aforementioned approach from various parameters. It is envisaged that the proposed evaluation scheme lays a platform for users to assess the recommender systems for their ease and reliable online shopping.

[1]  Giuseppe Carenini,et al.  Interactive multimedia summaries of evaluative text , 2006, IUI '06.

[2]  Sean M. McNee,et al.  Accurate is not always good: How Accuracy Metrics have hurt Recommender Systems , 2006 .

[3]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[4]  Guy Shani,et al.  Evaluating Recommendation Systems , 2011, Recommender Systems Handbook.

[5]  Saudi Arabia,et al.  PRO-MINING: PRODUCT RECOMMENDATION USING WEB-BASED OPINION MINING , 2013 .

[6]  Mouzhi Ge,et al.  Beyond accuracy: evaluating recommender systems by coverage and serendipity , 2010, RecSys '10.

[7]  Nesar Ahmad,et al.  Web search enhancement by mining user actions , 2007, Inf. Sci..

[8]  Rashid Ali,et al.  User Feedback Based Evaluation of a Product Recommendation System Using Rank Aggregation Method , 2014, ISI.

[9]  Sean M. McNee,et al.  On the recommending of citations for research papers , 2002, CSCW '02.

[10]  M. Matteucci,et al.  An Evaluation Methodology for Collaborative Recommender Systems , 2008, 2008 International Conference on Automated Solutions for Cross Media Content and Multi-Channel Distribution.

[11]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[12]  Rashid Ali,et al.  Book recommendation system using opinion mining technique , 2013, 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[13]  Lior Rokach,et al.  Recommender Systems Handbook , 2010 .

[14]  Jöran Beel,et al.  Sponsored vs. Organic (Research Paper) Recommendations and the Impact of Labeling , 2013, TPDL.

[15]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[16]  Sabine Bergler,et al.  Mining WordNet for a Fuzzy Sentiment: Sentiment Tag Extraction from WordNet Glosses , 2006, EACL.

[17]  Dietmar Jannach,et al.  What Recommenders Recommend - An Analysis of Accuracy, Popularity, and Sales Diversity Effects , 2013, UMAP.

[18]  Guy Shani,et al.  A Survey of Accuracy Evaluation Metrics of Recommendation Tasks , 2009, J. Mach. Learn. Res..

[19]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[20]  Andrew Turpin,et al.  Why batch and user evaluations do not give the same results , 2001, SIGIR '01.

[21]  Rashid Ali,et al.  User feedback scoring and evaluation of a product recommendation system , 2014, 2014 Seventh International Conference on Contemporary Computing (IC3).

[22]  Félix Hernández-del-Olmo,et al.  Evaluation of recommender systems: A new approach , 2008, Expert Syst. Appl..

[23]  Jöran Beel,et al.  The Impact of Demographics (Age and Gender) and Other User-Characteristics on Evaluating Recommender Systems , 2013, TPDL.

[24]  M. M. Sufyan Beg.  A subjective measure of web search quality , 2005, Inf. Sci..

[25]  Andreas Nürnberger,et al.  Research paper recommender system evaluation: a quantitative literature survey , 2013, RepSys '13.

[26]  Andrew Turpin,et al.  Do batch and user evaluations give the same results? , 2000, SIGIR '00.