Federated Multi-view Matrix Factorization for Personalized Recommendations

We introduce the federated multi-view matrix factorization method that extends the federated learning framework to matrix factorization with multiple data sources. Our method is able to learn the multi-view model without transferring the user's personal data to a central server. As far as we are aware this is the first federated model to provide recommendations using multi-view matrix factorization. The model is rigorously evaluated on three datasets on production settings. Empirical validation confirms that federated multi-view matrix factorization outperforms simpler methods that do not take into account the multi-view structure of the data, in addition, it demonstrates the usefulness of the proposed method for the challenging prediction tasks of cold-start federated recommendations.

[1]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[2]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[3]  Yoav Shoham,et al.  Fab: content-based, collaborative recommendation , 1997, CACM.

[4]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[5]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[6]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[7]  Dennis M. Wilkinson,et al.  Large-Scale Parallel Collaborative Filtering for the Netflix Prize , 2008, AAIM.

[8]  Geoffrey J. Gordon,et al.  Relational learning via collective matrix factorization , 2008, KDD.

[9]  Luo Si,et al.  Matrix co-factorization for recommendation with rich side information and implicit feedback , 2011, HetRec '11.

[10]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[11]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[12]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[13]  Inderjit S. Dhillon,et al.  Parallel matrix factorization for recommender systems , 2014, Knowl. Inf. Syst..

[14]  Krister Wennerberg,et al.  Integrative and Personalized QSAR Analysis in Cancer by Kernelized Bayesian Matrix Factorization , 2014, J. Chem. Inf. Model..

[15]  Samuel Kaski,et al.  Kernelized Bayesian Matrix Factorization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[17]  Sonali Agarwal,et al.  A Complete Survey on Software Architectural Styles and Patterns , 2015 .

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Samuel Kaski,et al.  Group Factor Analysis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Samuel Kaski,et al.  Sparse group factor analysis for biclustering of multiple data sources , 2015, Bioinform..

[21]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[22]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[23]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[24]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[25]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[26]  Zhenguo Li,et al.  Federated Meta-Learning with Fast Convergence and Efficient Communication , 2018, 1802.07876.

[27]  Zhenguo Li,et al.  Federated Meta-Learning for Recommendation , 2018, ArXiv.

[28]  David Cortes.  Cold-start recommendations in Collective Matrix Factorization , 2018, ArXiv.

[29]  Yang Liu,et al.  Secure Federated Transfer Learning , 2018, ArXiv.

[30]  Hans Hallez,et al.  Towards Privacy-preserving Mobile Applications with Federated Learning: The Case of Matrix Factorization (poster) , 2019, MobiSys.

[31]  Marco Scavuzzo,et al.  A Simple and Efficient Federated Recommender System , 2019, BDCAT.

[32]  Qiang Yang,et al.  Federated Machine Learning , 2019, ACM Trans. Intell. Syst. Technol..

[33]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[34]  Kuan Eeik Tan,et al.  Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System , 2019, ArXiv.

[35]  Samuel Kaski,et al.  Scalable Probabilistic Matrix Factorization with Graph-Based Priors , 2019, AAAI.

[36]  Tianjian Chen,et al.  A Secure Federated Transfer Learning Framework , 2020, IEEE Intelligent Systems.

[37]  Kai Chen,et al.  Secure Federated Matrix Factorization , 2019, IEEE Intelligent Systems.

[38]  Bingsheng He,et al.  A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection , 2019, IEEE Transactions on Knowledge and Data Engineering.