Toward Discriminating and Synthesizing Motion Traces Using Deep Probabilistic Generative Models

Mining knowledge from human mobility, such as discriminating motion traces left by different anonymous users, also known as the trajectory-user linking (TUL) problem, is an important task in many applications requiring location-based services (LBSs). However, it inevitably raises an issue that may be aggravated by TUL, i.e., how to defend against location attacks (e.g., deanonymization and location recovery). In this work, we present a Semisupervised Trajectory- User Linking model with Interpretable representation and Gaussian mixture prior (STULIG)--a novel deep probabilistic framework for jointly learning disentangled representation of user trajectories in a semisupervised manner and tackling the location recovery problem. STULIG characterizes multiple latent aspects of human trajectories and their labels into separate latent variables, which can be then used to interpret user check-in styles and improve the performance of trace classification. It can also generate synthetic yet plausible trajectories, thus protecting users' actual locations while preserving the meaningful mobility information for various machine learning tasks. We analyze and evaluate STULIG's ability to learn disentangled representations, discriminating human traces and generating realistic motions on several real-world mobility data sets. As demonstrated by extensive experimental evaluations, in addition to outperforming the state-of-the-art methods, our method provides intuitive explanations of the classification and generation and sheds lights on the interpretable mobility mining.

[1]  Joelle Pineau,et al.  Language GANs Falling Short , 2018, ICLR.

[2]  Max Welling,et al.  VAE with a VampPrior , 2017, AISTATS.

[3]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[4]  Ravi Jain,et al.  Evaluating location predictors with extensive Wi-Fi mobility data , 2003, IEEE INFOCOM 2004.

[5]  Qionghai Dai,et al.  Cross-Modality Bridging and Knowledge Transferring for Image Understanding , 2019, IEEE Transactions on Multimedia.

[6]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[7]  Stefan Bauer,et al.  Disentangling Factors of Variations Using Few Labels , 2020, ICLR.

[8]  David S. Rosenblum,et al.  A Non-Parametric Generative Model for Human Trajectories , 2018, IJCAI.

[9]  Hao Liu,et al.  Learning User's Intrinsic and Extrinsic Interests for Point-of-Interest Recommendation: A Unified Approach , 2017, IJCAI.

[10]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[11]  Emiliano De Cristofaro,et al.  Differentially Private Mixture of Generative Neural Networks , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[12]  Yining Chen,et al.  Weakly Supervised Disentanglement with Guarantees , 2020, ICLR.

[13]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[14]  Maria Luisa Damiani,et al.  Efficient Access to Temporally Overlaying Spatial and Textual Trajectories , 2016, 2016 17th IEEE International Conference on Mobile Data Management (MDM).

[15]  Jean-François Paiement,et al.  A Generative Model of Urban Activities from Cellular Data , 2018, IEEE Transactions on Intelligent Transportation Systems.

[16]  Carl A. Gunter,et al.  Plausible Deniability for Privacy-Preserving Data Synthesis , 2017, Proc. VLDB Endow..

[17]  Frank Hansen,et al.  Jensen's Operator Inequality for Functions of Several Variables , 2000 .

[18]  Gao Cong,et al.  An Experimental Evaluation of Point-of-interest Recommendation in Location-based Social Networks , 2017, Proc. VLDB Endow..

[19]  Craig MacDonald,et al.  A Deep Recurrent Collaborative Filtering Framework for Venue Recommendation , 2017, CIKM.

[20]  Frank D. Wood,et al.  Learning Disentangled Representations with Semi-Supervised Deep Generative Models , 2017, NIPS.

[21]  Xing Xie,et al.  GeoMF: joint geographical modeling and matrix factorization for point-of-interest recommendation , 2014, KDD.

[22]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[23]  Chao Zhang,et al.  DeepMove: Predicting Human Mobility with Attentional Recurrent Networks , 2018, WWW.

[24]  Tara N. Sainath,et al.  An Analysis of "Attention" in Sequence-to-Sequence Models , 2017, INTERSPEECH.

[25]  Qiang Gao,et al.  Predicting Human Mobility via Variational Attention , 2019, WWW.

[26]  Yongdong Zhang,et al.  STAT: Spatial-Temporal Attention Mechanism for Video Captioning , 2020, IEEE Transactions on Multimedia.

[27]  Ali Razavi,et al.  Preventing Posterior Collapse with delta-VAEs , 2019, ICLR.

[28]  Zhiyuan Liu,et al.  A Neural Network Approach to Joint Modeling Social Networks and Mobile Trajectories , 2016, ArXiv.

[29]  Qiang Gao,et al.  Trajectory-User Linking via Variational AutoEncoder , 2018, IJCAI.

[30]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[31]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[32]  Tieniu Tan,et al.  Predicting the Next Location: A Recurrent Model with Spatial and Temporal Contexts , 2016, AAAI.

[33]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Cheng Soon Ong,et al.  Learning Points and Routes to Recommend Trajectories , 2016, CIKM.

[35]  Pieter Abbeel,et al.  Variational Lossy Autoencoder , 2016, ICLR.

[36]  Alexander M. Rush,et al.  Dilated Convolutions for Modeling Long-Distance Genomic Dependencies , 2017, bioRxiv.

[37]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[38]  Jiawei Han,et al.  Bridging Collaborative Filtering and Semi-Supervised Learning: A Neural Approach for POI Recommendation , 2017, KDD.

[39]  Bernhard Schölkopf,et al.  Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations , 2018, ICML.

[40]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[41]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[42]  Qiang Gao,et al.  Trajectory-based social circle inference , 2018, SIGSPATIAL/GIS.

[43]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[44]  Craig MacDonald,et al.  A Contextual Attention Recurrent Architecture for Context-Aware Venue Recommendation , 2018, SIGIR.

[45]  Alexander A. Alemi,et al.  Fixing a Broken ELBO , 2017, ICML.

[46]  Ying Tan,et al.  Semisupervised Text Classification by Variational Autoencoder , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Kunpeng Zhang,et al.  Adversarial Point-of-Interest Recommendation , 2019, WWW.

[48]  Xiaoming Fu,et al.  Trajectory Recovery From Ash: User Privacy Is NOT Preserved in Aggregated Mobility Data , 2017, WWW.

[49]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[50]  Qiang Gao,et al.  Identifying Human Mobility via Trajectory Embeddings , 2017, IJCAI.

[51]  Michael R. Lyu,et al.  Where You Like to Go Next: Successive Point-of-Interest Recommendation , 2013, IJCAI.

[52]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[53]  Moritz Hardt,et al.  Stable Recurrent Models , 2018, ICLR.

[54]  Dana H. Brooks,et al.  Structured Disentangled Representations , 2018, AISTATS.

[55]  Yann Dauphin,et al.  Convolutional Sequence to Sequence Learning , 2017, ICML.

[56]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[57]  Murray Shanahan,et al.  Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders , 2016, ArXiv.

[58]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[59]  Roger B. Grosse,et al.  Isolating Sources of Disentanglement in Variational Autoencoders , 2018, NeurIPS.

[60]  Yongdong Zhang,et al.  A Fast Uyghur Text Detector for Complex Background Images , 2018, IEEE Transactions on Multimedia.

[61]  Gao Cong,et al.  Graph-based Point-of-interest Recommendation with Geographical and Temporal Influences , 2014, CIKM.

[62]  Xuan Song,et al.  DeepTransport: Prediction and Simulation of Human Mobility and Transportation Mode at a Citywide Level , 2016, IJCAI.

[63]  Li Zhao,et al.  Learning Structured Representation for Text Classification via Reinforcement Learning , 2018, AAAI.