Reinforced Imitative Graph Representation Learning for Mobile User Profiling: An Adversarial Training Perspective

In this paper, we study the problem of mobile user profiling, which is a critical component for quantifying users’ characteristics in the human mobility modeling pipeline. Human mobility is a sequential decision-making process dependent on the users’ dynamic interests. With accurate user profiles, the predictive model can perfectly reproduce users’ mobility trajectories. In the reverse direction, once the predictive model can imitate users’ mobility patterns, the learned user profiles are also optimal. Such intuition motivates us to propose an imitation-based mobile user profiling framework by exploiting reinforcement learning, in which the agent is trained to precisely imitate users’ mobility patterns for optimal user profiles. Specifically, the proposed framework includes two modules: (1) representation module, that produces state combining user profiles and spatio-temporal context in real-time; (2) imitation module, where Deep Q-network (DQN) imitates the user behavior (action) based on the state that is produced by the representation module. However, there are two challenges in running the framework effectively. First, -greedy strategy in DQN makes use of the explorationexploitation trade-off by randomly pick actions with the probability. Such randomness feeds back to the representation module, causing the learned user profiles unstable. To solve the problem, we propose an adversarial training strategy to guarantee the robustness of the representation module. Second, the representation module updates users’ profiles in an incremental manner, requiring integrating the temporal effects of user profiles. Inspired by Long-short Term Memory (LSTM), we introduce a gated mechanism to incorporate new and old user characteristics into the user profile. In the experiment, we evaluate our proposed framework on real-world datasets. The extensive experimental results validate the superiority of our method comparing to baseline algorithms.

[1]  Paul Covington,et al.  Deep Neural Networks for YouTube Recommendations , 2016, RecSys.

[2]  Ling Chen,et al.  Spatial-Aware Hierarchical Collaborative Deep Learning for POI Recommendation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[3]  Talel Abdessalem,et al.  POI Recommendation: Towards Fused Matrix Factorization with Geographical and Temporal Influences , 2015, RecSys.

[4]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[5]  Daniel Kifer,et al.  Non-Stationary Model for Crime Rate Inference Using Modern Urban Data , 2019, IEEE Transactions on Big Data.

[6]  Yiteng Pan,et al.  Learning social representations with deep autoencoder for recommender system , 2020, World Wide Web.

[7]  Charu C. Aggarwal,et al.  Ensemble-Spotting: Ranking Urban Vibrancy via POI Embedding with Multi-view Spatial Graphs , 2018, SDM.

[8]  Hui Xiong,et al.  Adversarial Substructured Representation Learning for Mobile User Profiling , 2019, KDD.

[9]  Hui Xiong,et al.  An Integrated Model for Crime Prediction Using Temporal and Spatial Factors , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[10]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[11]  Michael R. Lyu,et al.  STELLAR: Spatial-Temporal Latent Ranking Model for Successive POI Recommendation , 2018 .

[12]  Dhruv Grewal,et al.  Mobile Advertising: A Framework and Research Agenda , 2016 .

[13]  Yi Tay,et al.  Deep Learning based Recommender System: A Survey and New Perspectives , 2018 .

[14]  Daqing Zhang,et al.  Modeling User Activity Preference by Leveraging User Spatial Temporal Characteristics in LBSNs , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Liang Zhang,et al.  Deep reinforcement learning for page-wise recommendations , 2018, RecSys.

[16]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[17]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[18]  Kunpeng Liu,et al.  Incremental Mobile User Profiling: Reinforcement Learning with Spatial Knowledge Graph for Modeling Event Streams , 2020, KDD.

[19]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.