Research on the Matthews Correlation Coefficients Metrics of Personalized Recommendation Algorithm Evaluation

The personalized recommendation systems could better improve the personalized service for network user and alleviate the problem of information overload in the Internet. As we all know, the key point of being a successful recommendation system is the performance of recommendation algorithm. When scholars put forward some new recommendation algorithms, they claim that the new algorithms have been improved in some respects, better than previous algorithm. So we need some evaluation metrics to evaluate the algorithm performance. Due to the scholar didn’t fully understand the evaluation mechanism of recommendation algorithms. They mainly emphasized some specific evaluation metrics like Accuracy, Diversity. What’s more, the academia did not establish a complete and unified assessment of recommendation algorithms evaluation system which is credibility to do the work of recommendation evaluation. So how to do this work objective and reasonable is still a challengeable task. In this article, we discussed the present evaluation metrics with its respective advantages and disadvantages. Then, we put forward to use the Matthews Correlation Coefficient to evaluate the recommendation algorithm’s performance. All this based on an open source projects called mahout which provides a rich set of components to construct the classic recommendation algorithm. The results of the experiments show that the applicability of Matthews correlation coefficient in the relative evaluation work of recommendation algorithm.

[1]  Yi-Cheng Zhang,et al.  Personalized Recommendation via Integrated Diffusion on User-Item-Tag Tripartite Graphs , 2009, ArXiv.

[2]  John A. Swets,et al.  Effectiveness of information retrieval methods , 1969 .

[3]  Yoav Shoham,et al.  Fab: content-based, collaborative recommendation , 1997, CACM.

[4]  Yi-Cheng Zhang,et al.  Effect of initial configuration on network-based recommendation , 2007, 0711.2506.

[5]  John Riedl,et al.  Analysis of recommendation algorithms for e-commerce , 2000, EC '00.

[6]  Michael J. Pazzani,et al.  Learning Collaborative Information Filters , 1998, ICML.

[7]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[8]  Foreword and Editorial International Journal of Hybrid Information Technology , 2022 .

[9]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[10]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[11]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[12]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[13]  Runran Liu,et al.  Personal recommendation via modified collaborative filtering , 2008, Physica A: Statistical Mechanics and its Applications.

[14]  B. Matthews.  Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[15]  Kirsten Swearingen,et al.  Beyond Algorithms: An HCI Perspective on Recommender Systems , 2001 .

[16]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[17]  Michael J. Pazzani,et al.  Learning and Revising User Profiles: The Identification of Interesting Web Sites , 1997, Machine Learning.

[18]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[19]  William W. Cohen,et al.  Recommendation as Classification: Using Social and Content-Based Information in Recommendation , 1998, AAAI/IAAI.

[20]  Masataka Goto,et al.  An Efficient Hybrid Music Recommender System Using an Incrementally Trainable Probabilistic Generative Model , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  Tao Li,et al.  Recommendation model based on opinion diffusion , 2007, ArXiv.

[22]  Yi-Cheng Zhang,et al.  Heat conduction process on community networks as a recommendation model. , 2007, Physical review letters.