C3DRec: Cloud-Client Cooperative Deep Learning for Temporal Recommendation in the Post-GDPR Era

Mobile devices enable users to retrieve information at any time and any place. Considering the occasional requirements and fragmentation usage pattern of mobile users, temporal recommendation techniques are proposed to improve the efficiency of information retrieval on mobile devices by means of accurately recommending items via learning temporal interests with short-term user interaction behaviors. However, the enforcement of privacy-preserving laws and regulations, such as the General Data Protection Regulation (GDPR), may overshadow the successful practice of temporal recommendation. The reason is that state-of-the-art recommendation systems require to gather and process the user data in centralized servers but the interaction behaviors data used for temporal recommendation are usually non-transactional data that are not allowed to gather without the explicit permission of users according to GDPR. As a result, if users do not permit services to gather their interaction behaviors data, the temporal recommendation fails to work. To realize the temporal recommendation in the postGDPR era, this paper proposes C3DRec , a cloud-client cooperative deep learning framework of mining interaction behaviors for recommendation while preserving user privacy.C3DRec constructs a global recommendation model on centralized servers using data collected before GDPR and fine-tunes the model directly on individual local devices using data collected after GDPR. We design two modes to accomplish the recommendation, i.e., pull mode where candidate items are pulled down onto the devices and fed into the local model to get recommended items, and push mode where the output of the local model is pushed onto the server and combined with candidate items to get recommended ones. Evaluation results show that C3DRec achieves comparable recommendation accuracy to the centralized approaches, with minimal privacy concern.

[1]  Harald Steck,et al.  Gaussian Ranking by Matrix Factorization , 2015, RecSys.

[2]  Tsvi Kuflik,et al.  Incorporating Dwell Time in Session-Based Recommendations with Recurrent Neural Networks , 2017, RecTemp@RecSys.

[3]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[4]  Dietmar Jannach,et al.  When Recurrent Neural Networks meet the Neighborhood for Session-Based Recommendation , 2017, RecSys.

[5]  Yu He,et al.  The YouTube video recommendation system , 2010, RecSys '10.

[6]  De-Chuan Zhan,et al.  Adaptive Deep Models for Incremental Learning: Considering Capacity Scalability and Sustainability , 2019, KDD.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[9]  Martin Wattenberg,et al.  TensorFlow.js: Machine Learning for the Web and Beyond , 2019, MLSys.

[10]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[11]  Xuanzhe Liu,et al.  DeepCache: Principled Cache for Mobile Deep Vision , 2017, MobiCom.

[12]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[13]  Feng Qian,et al.  DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning , 2017, IEEE Transactions on Mobile Computing.

[14]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[15]  Kun Gai,et al.  Learning Tree-based Deep Model for Recommender Systems , 2018, KDD.

[16]  Kaigui Bian,et al.  Characterizing Smartphone Usage Patterns from Millions of Android Users , 2015, Internet Measurement Conference.

[17]  Ying Zhang,et al.  Programming Situational Mobile Web Applications with Cloud-Mobile Convergence: An Internetware-Oriented Approach , 2019, IEEE Transactions on Services Computing.

[18]  John C. Tang,et al.  Mobile taskflow in context: a screenshot study of smartphone usage , 2010, CHI.

[19]  Gang Wang,et al.  Modeling Spatio-Temporal App Usage for a Large User Population , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[20]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[21]  Alexandros Karatzoglou,et al.  Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations , 2016, RecSys.

[22]  Yang Song,et al.  Multi-Rate Deep Learning for Temporal Recommendation , 2016, SIGIR.

[23]  Jing Cai,et al.  Enlister: baidu's recommender system for the biggest chinese Q&A website , 2012, RecSys.

[24]  Talel Abdessalem,et al.  Dynamic Local Models for Online Recommendation , 2018, WWW.

[25]  Yiling Lou,et al.  An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).

[26]  Masamichi Shimosaka,et al.  Early Destination Prediction with Spatio-temporal User Behavior Patterns , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[27]  Xuanzhe Liu,et al.  Understanding Diverse Usage Patterns from Large-Scale Appstore-Service Profiles , 2017, IEEE Transactions on Software Engineering.

[28]  Xiaolin Li,et al.  Inferring Correlation between User Mobility and App Usage in Massive Coarse-grained Data Traces , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[29]  Xuanzhe Liu,et al.  A First Look at Deep Learning Apps on Smartphones , 2018, WWW.

[30]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[31]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[32]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[33]  Alex Beutel,et al.  Recurrent Recommender Networks , 2017, WSDM.

[34]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[35]  Martha Larson,et al.  CLiMF: learning to maximize reciprocal rank with collaborative less-is-more filtering , 2012, RecSys.

[36]  Chao Huang,et al.  Privacy-preserving Cross-domain Location Recommendation , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[37]  Suyog Gupta,et al.  To prune, or not to prune: exploring the efficacy of pruning for model compression , 2017, ICLR.

[38]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[39]  Shahriar Nirjon,et al.  Intermittent Learning: On-Device Machine Learning on Intermittently Powered System , 2019 .

[40]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[41]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[42]  Tao Xie,et al.  A comprehensive study on challenges in deploying deep learning based software , 2020, ESEC/SIGSOFT FSE.

[43]  Alexandros Karatzoglou,et al.  Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks , 2017, RecSys.

[44]  Xuanzhe Liu,et al.  Approximate Query Processing on Autonomous Cameras , 2019, ArXiv.

[45]  Xuanzhe Liu,et al.  Through a Gender Lens: Learning Usage Patterns of Emojis from Large-Scale Android Users , 2017, WWW.

[46]  Yukihiro Tagami,et al.  Embedding-based News Recommendation for Millions of Users , 2017, KDD.

[47]  Xuanzhe Liu,et al.  Moving Deep Learning into Web Browser: How Far Can We Go? , 2019, WWW.

[48]  João Gama,et al.  A survey on concept drift adaptation , 2014, ACM Comput. Surv..