A Privacy-Preserving Distributed Contextual Federated Online Learning Framework with Big Data Support in Social Recommender Systems

Nowadays the booming demand of big data analytics and the constraints of computational ability and network bandwidth have made it difficult for a stand-alone agent/service provider to provide suitable information for every user from the large volume online data within the time limited. To handle this challenge, a recommender system (RS) can call in a group of agents to collaborate to learn users' preference and taste, which is known as a distributed recommender system (DRS). In this paper, we propose a privacy-preserving DRS, where each service provider (SP) is modeled as a distributed context-aware online learner. SPs collaborate to make personalized recommendations by learning users' preferences based on the user context and users' previous behaviors. To support big data analytics, we establish an item-cluster tree from top to the bottom to handle increasing datasets. We further consider the structure of social network and propose an adaptive algorithm to reduce performance loss. Theoretical analysis show that our proposal can achieve sublinear regret and differential privacy of both SPs and users. Numerical results validate that our new framework can support increasing big datasets while striking a balance between privacy-preserving level (PPL) and prediction accuracy.

[1]  Roshan Shariff,et al.  Differentially Private Contextual Linear Bandits , 2018, NeurIPS.

[2]  Omer Tene,et al.  Big Data for All: Privacy and User Control in the Age of Analytics , 2012 .

[3]  Vitaly Shmatikov,et al.  How To Break Anonymity of the Netflix Prize Dataset , 2006, ArXiv.

[4]  Zhen Lin,et al.  Context-Aware Collaborative Topic Regression with Social Matrix Factorization for Recommender Systems , 2014, AAAI.

[5]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[6]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[7]  H. Vincent Poor,et al.  From Technological Networks to Social Networks , 2013, IEEE Journal on Selected Areas in Communications.

[8]  Zekeriya Erkin,et al.  Generating Private Recommendations Efficiently Using Homomorphic Encryption and Data Packing , 2012, IEEE Transactions on Information Forensics and Security.

[9]  David A. Shamma,et al.  The New Data and New Challenges in Multimedia Research , 2015, ArXiv.

[10]  Mehrbakhsh Nilashi,et al.  Collaborative filtering recommender systems , 2013 .

[11]  Hossein Saiedian,et al.  A Distributed Recommender System Architecture , 2009, Software Engineering Research and Practice.

[12]  Yan Zhang,et al.  Mobile Edge Computing: A Survey , 2018, IEEE Internet of Things Journal.

[13]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[14]  Lalita Sharma,et al.  A Survey of Recommendation System: Research Challenges , 2013 .

[15]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[16]  Nicholas Jing Yuan,et al.  Scalable Content-Aware Collaborative Filtering for Location Recommendation , 2018, IEEE Transactions on Knowledge and Data Engineering.

[17]  Kunal Talwar,et al.  Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[18]  Sanjeev R. Kulkarni,et al.  Beyond personalization and anonymity: towards a group-based recommender system , 2014, SAC.

[19]  Wenyu Zhang,et al.  An Approach for Building Efficient and Accurate Social Recommender Systems Using Individual Relationship Networks , 2017, IEEE Transactions on Knowledge and Data Engineering.

[20]  Bankim Patel,et al.  Big Data Analysis: Recommendation System with Hadoop Framework , 2015, 2015 IEEE International Conference on Computational Intelligence & Communication Technology.

[21]  Vitaly Shmatikov,et al.  The cost of privacy: destruction of data-mining utility in anonymized data publishing , 2008, KDD.

[22]  Gediminas Adomavicius,et al.  Context-aware recommender systems , 2008, RecSys '08.

[23]  Aleksandrs Slivkins,et al.  Contextual Bandits with Similarity Information , 2009, COLT.

[24]  Mihaela van der Schaar,et al.  Online Learning in Large-Scale Contextual Recommender Systems , 2016, IEEE Transactions on Services Computing.

[25]  Hao Yang,et al.  A Survey of Collaborative Filtering Algorithms for Social Recommender Systems , 2016, 2016 12th International Conference on Semantics, Knowledge and Grids (SKG).

[26]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[27]  Mihaela van der Schaar,et al.  Distributed online Big Data classification using context information , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[28]  Charu C. Aggarwal,et al.  On k-Anonymity and the Curse of Dimensionality , 2005, VLDB.

[29]  Rémi Munos,et al.  Stochastic Simultaneous Optimistic Optimization , 2013, ICML.

[30]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[31]  Alessandro Lazaric,et al.  Online Stochastic Optimization under Correlated Bandit Feedback , 2014, ICML.

[32]  Mihaela van der Schaar,et al.  Distributed Online Learning in Social Recommender Systems , 2013, IEEE Journal of Selected Topics in Signal Processing.

[33]  Murtaza Haider,et al.  Beyond the hype: Big data concepts, methods, and analytics , 2015, Int. J. Inf. Manag..

[34]  Chien Chin Chen,et al.  An effective recommendation method for cold start new users using trust and distrust networks , 2013, Inf. Sci..

[35]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[36]  Philippe Cudré-Mauroux,et al.  Privacy-Preserving Social Media Data Publishing for Personalized Ranking-Based Recommendation , 2019, IEEE Transactions on Knowledge and Data Engineering.

[37]  Elaine Shi,et al.  Private and Continual Release of Statistics , 2010, TSEC.

[38]  Pasquale Lops,et al.  Content-based Recommender Systems: State of the Art and Trends , 2011, Recommender Systems Handbook.

[39]  Mi-Yen Yeh,et al.  A General Framework for Implicit and Explicit Social Recommendation , 2018, IEEE Transactions on Knowledge and Data Engineering.

[40]  Moni Naor,et al.  Differential privacy under continual observation , 2010, STOC '10.

[41]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[42]  Yanjiao Chen,et al.  Privacy-Preserving Collaborative Model Learning: The Case of Word Vector Training , 2018, IEEE Transactions on Knowledge and Data Engineering.

[43]  Jouni Markkula,et al.  EU General Data Protection Regulation: Changes and implications for personal data collecting companies , 2017, Comput. Law Secur. Rev..

[44]  Neil Yorke-Smith,et al.  TrustSVD: Collaborative Filtering with Both the Explicit and Implicit Influence of User Trust and of Item Ratings , 2015, AAAI.

[45]  Qian Wang,et al.  Differentially Private Distributed Online Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[46]  Ting Yu,et al.  A Privacy-Preserving Framework for Personalized, Social Recommendations , 2014, EDBT.

[47]  Eli Upfal,et al.  Multi-Armed Bandits in Metric Spaces ∗ , 2008 .

[48]  Atilla Eryilmaz,et al.  Multi-armed bandits in the presence of side observations in social networks , 2013, 52nd IEEE Conference on Decision and Control.

[49]  Martin Pál,et al.  Contextual Multi-Armed Bandits , 2010, AISTATS.

[50]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[51]  Gediminas Adomavicius,et al.  Incorporating contextual information in recommender systems using a multidimensional approach , 2005, TOIS.

[52]  Michal Valko,et al.  Adaptive black-box optimization got easier: HCT only needs local smoothness , 2018, EWRL 2018.