Robust and Accurate Representation Learning for High-dimensional and Sparse Matrices in Recommender Systems

How to accurately represent a high-dimensional and sparse (HiDS) user-item rating matrix is a crucial issue in implementing a recommender system. A latent factor (LF) model is one of the most popular and successful approaches to address this issue. It is developed by minimizing the errors between the observed entries and the estimated ones on an HiDS matrix. Current studies commonly employ L2-norm to minimize the errors because it has a smooth gradient, making a resultant LF model can accurately represent an HiDS matrix. As is well known, however, L2-norm is very sensitive to the outlier data or called unreliable ratings in the context of the recommender system. Unfortunately, the unreliable ratings often exist in an HiDS matrix due to some malicious users. To address this issue, this paper proposes a Smooth L1-norm-oriented Latent Factor (SL1-LF) model. Its main idea is to employ smooth L1-norm rather than L2-norm to minimize the errors, making it have both high robustness and accuracy in representing an HiDS matrix. Experimental results on four HiDS matrices generated by industrial recommender systems demonstrate that the proposed SL1-LF model is robust to the outlier data and has significantly higher prediction accuracy than state-of-the-art models for the missing data of an HiDS matrix.

[1]  Tat-Seng Chua,et al.  Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks , 2017, IJCAI.

[2]  Meina Song,et al.  Context-aware probabilistic matrix factorization modeling for point-of-interest recommendation , 2017, Neurocomputing.

[3]  Guoyin Wang,et al.  A Data-Aware Latent Factor Model for Web Service QoS Prediction , 2019, PAKDD.

[4]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[5]  Timos Sellis,et al.  Big data analytics in telecommunications: literature review and architecture recommendations , 2020, IEEE/CAA Journal of Automatica Sinica.

[6]  Hao Wu,et al.  Dual-regularized matrix factorization with deep neural networks for recommender systems , 2018, Knowl. Based Syst..

[7]  Piji Li,et al.  Neural Rating Regression with Abstractive Tips Generation for Recommendation , 2017, SIGIR.

[8]  Mingsheng Shang,et al.  DCCR: Deep Collaborative Conjunctive Recommender for Rating Prediction , 2019, IEEE Access.

[9]  Hong Yang,et al.  Artificial intelligence applications in the development of autonomous vehicles: a survey , 2020, IEEE/CAA Journal of Automatica Sinica.

[10]  Enhong Chen,et al.  Confidence-Aware Matrix Factorization for Recommender Systems , 2018, AAAI.

[11]  Guoyin Wang,et al.  Water eutrophication evaluation based on semi-supervised classification: A case study in Three Gorges Reservoir , 2017 .

[12]  Scott Sanner,et al.  AutoRec: Autoencoders Meet Collaborative Filtering , 2015, WWW.

[13]  Yi Tay,et al.  Deep Learning based Recommender System: A Survey and New Perspectives , 2018 .

[14]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[15]  R. Glynn,et al.  The Wilcoxon Signed Rank Test for Paired Comparisons of Clustered Data , 2006, Biometrics.

[16]  Guoyin Wang,et al.  Self-training semi-supervised classification based on density peaks of data , 2018, Neurocomputing.

[17]  Donghyun Kim,et al.  Convolutional Matrix Factorization for Document Context-Aware Recommendation , 2016, RecSys.

[18]  Xiaoyu Du,et al.  Fast Matrix Factorization With Nonuniform Weights on Missing Data , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Jia Chen,et al.  Randomized latent factor model for high-dimensional and sparse matrices from industrial applications , 2018, 2018 IEEE 15th International Conference on Networking, Sensing and Control (ICNSC).

[20]  Yihong Gong,et al.  Fast nonparametric matrix factorization for large-scale collaborative filtering , 2009, SIGIR.

[21]  Ji Feng,et al.  Deep Forest: Towards An Alternative to Deep Neural Networks , 2017, IJCAI.

[22]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[23]  Dimitri P. Bertsekas,et al.  Feature-based aggregation and deep reinforcement learning: a survey and some new implementations , 2018, IEEE/CAA Journal of Automatica Sinica.

[24]  Greg Linden,et al.  Two Decades of Recommender Systems at Amazon.com , 2017, IEEE Internet Computing.

[25]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[26]  Di Wu,et al.  A Highly Accurate Framework for Self-Labeled Semisupervised Classification in Industrial Applications , 2018, IEEE Transactions on Industrial Informatics.

[27]  Mohammad Ali Abbasi,et al.  Trust-Aware Recommender Systems , 2014 .

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[29]  Guoyin Wang,et al.  A Posterior-Neighborhood-Regularized Latent Factor Model for Highly Accurate Web Service QoS Prediction , 2022, IEEE Transactions on Services Computing.

[30]  Luis Martínez,et al.  Opinion Dynamics-Based Group Recommender Systems , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[32]  Lei Zheng,et al.  Joint Deep Modeling of Users and Items Using Reviews for Recommendation , 2017, WSDM.

[33]  Di Wu,et al.  A Generalized and Fast-converging Non-negative Latent Factor Model for Predicting User Preferences in Recommender Systems , 2020, WWW.

[34]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Zhenyu He,et al.  Similarity-Maintaining Privacy Preservation and Location-Aware Low-Rank Matrix Factorization for QoS Prediction Based Web Service Recommendation , 2021, IEEE Transactions on Services Computing.

[36]  Jin Xu,et al.  Enabling Kernel-Based Attribute-Aware Matrix Factorization for Rating Prediction , 2017, IEEE Transactions on Knowledge and Data Engineering.

[37]  Xindong Wu,et al.  Online Feature Selection with Capricious Streaming Features: A General Framework , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[38]  Guillaume Bouchard,et al.  Robust Bayesian Matrix Factorisation , 2011, AISTATS.

[39]  Vaclav Petricek,et al.  Recommender System for Online Dating Service , 2007, ArXiv.

[40]  Jongmoon Baik,et al.  Location-Based Web Service QoS Prediction via Preference Propagation to Address Cold Start Problem , 2021, IEEE Transactions on Services Computing.

[41]  Kay Chen Tan,et al.  Multiobjective Sparse Non-Negative Matrix Factorization , 2019, IEEE Transactions on Cybernetics.

[42]  Naomi S. Altman,et al.  Quantile regression , 2019, Nature Methods.

[43]  Anup Basu,et al.  Graph regularized Lp smooth non-negative matrix factorization for data representation , 2019, IEEE/CAA Journal of Automatica Sinica.

[44]  MengChu Zhou,et al.  A Deep Latent Factor Model for High-Dimensional and Sparse Matrices in Recommender Systems , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[45]  Zibin Zheng,et al.  QoS Prediction of Web Services Based on Two-Phase K-Means Clustering , 2015, 2015 IEEE International Conference on Web Services.

[46]  Jiandong Duan,et al.  Sparse Normalized Least Mean Absolute Deviation Algorithm Based on Unbiasedness Criterion for System Identification With Noisy Input , 2018, IEEE Access.

[47]  Mingsheng Shang,et al.  A Fast Non-Negative Latent Factor Model Based on Generalized Momentum Method , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[48]  Yi He,et al.  Toward Mining Capricious Data Streams: A Generative Approach , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.