mRHR: A Modified Reciprocal Hit Rank Metric for Ranking Evaluation of Multiple Preferences in Top-N Recommender Systems

Average reciprocal hit rank (ARHR) is a commonly used metric for ranking evaluation of top-n recommender systems. However, it suffers from an important shortcoming that it cannot be applied when the user has multiple preferences at a time. In order to overcome this problem, a modified version of ARHR metric is introduced and applied to grocery shopping domain by conducting a series of experiments on real-life data. The results show that the proposed measure is feasible for ranking evaluation of Top-N recommender systems in the cases where the users have multiple preferences at a time or a specific time interval.

[1]  Olivier Chapelle,et al.  Expected reciprocal rank for graded relevance , 2009, CIKM.

[2]  Mark Rosenstein,et al.  Recommending and evaluating choices in a virtual community of use , 1995, CHI '95.

[3]  John Riedl,et al.  Analysis of recommendation algorithms for e-commerce , 2000, EC '00.

[4]  George Karypis,et al.  SLIM: Sparse Linear Methods for Top-N Recommender Systems , 2011, 2011 IEEE 11th International Conference on Data Mining.

[5]  Tommaso Di Noia,et al.  Top-N recommendations from implicit feedback leveraging linked open data , 2013, IIR.

[6]  Alistair Moffat,et al.  Rank-biased precision for measurement of retrieval effectiveness , 2008, TOIS.

[7]  Rayid Ghani,et al.  Predicting customer shopping lists from point-of-sale purchase data , 2004, KDD.

[8]  Tovi Grossman,et al.  Design and evaluation of a command recommendation system for software applications , 2011, TCHI.

[9]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[10]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[11]  Roberto Turrin,et al.  Performance of recommender algorithms on top-n recommendation tasks , 2010, RecSys '10.

[12]  Zhao Kang,et al.  Top-N Recommendation with Novel Rank Approximation , 2016, SDM.

[13]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[14]  Jaana Kekäläinen,et al.  Binary and graded relevance in IR evaluations--Comparison of the effects on ranking of IR systems , 2005, Inf. Process. Manag..

[15]  Yong Yu,et al.  LorSLIM: Low Rank Sparse Linear Methods for Top-N Recommendations , 2014, 2014 IEEE International Conference on Data Mining.

[16]  Rayid Ghani,et al.  Building intelligent shopping assistants using individual consumer models , 2005, IUI '05.

[17]  Qiudan Li,et al.  A recommender system based on tag and time information for social tagging systems , 2011, Expert Syst. Appl..