Meta Learning with Relational Information for Short Sequences

This paper proposes a new meta-learning method -- named HARMLESS (HAwkes Relational Meta Learning method for Short Sequences) for learning heterogeneous point process models from a collection of short event sequence data along with a relational network. Specifically, we propose a hierarchical Bayesian mixture Hawkes process model, which naturally incorporates the relational information among sequences into point process modeling. Compared with existing methods, our model can capture the underlying mixed-community patterns of the relational network, which simultaneously encourages knowledge sharing among sequences and facilitates adaptively learning for each individual sequence. We further propose an efficient stochastic variational meta-EM algorithm, which can scale to large problems. Numerical experiments on both synthetic and real data show that HARMLESS outperforms existing methods in terms of predicting the future events.

[1]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[2]  Hong Yu,et al.  Meta Networks , 2017, ICML.

[3]  Shuang-Hong Yang,et al.  Mixture of Mutually Exciting Processes for Viral Diffusion , 2013, ICML.

[4]  Y. Ogata Seismicity Analysis through Point-process Modeling: A Review , 1999 .

[5]  Gregory R. Koch,et al.  Siamese Neural Networks for One-Shot Image Recognition , 2015 .

[6]  Le Song,et al.  Fake News Mitigation via Point Process Based Intervention , 2017, ICML.

[7]  R. Dahlhaus,et al.  Graphical Modeling for Multivariate Hawkes Processes with Nonparametric Link Functions , 2016, 1605.06759.

[8]  Hongyuan Zha,et al.  Correlated Cascades: Compete or Cooperate , 2015, AAAI.

[9]  Renaud Lambiotte,et al.  TiDeH: Time-Dependent Hawkes Process for Predicting Retweet Dynamics , 2016, ICWSM.

[10]  Hongyuan Zha,et al.  A Dirichlet Mixture Model of Hawkes Processes for Event Sequence Clustering , 2017, NIPS.

[11]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[12]  J. Schulman,et al.  Reptile: a Scalable Metalearning Algorithm , 2018 .

[13]  Andrea L. Bertozzi,et al.  Modeling E-mail Networks and Inferring Leadership Using Self-Exciting Point Processes , 2016 .

[14]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[15]  Jure Leskovec,et al.  Motifs in Temporal Networks , 2016, WSDM.

[16]  Le Song,et al.  NetCodec: Community Detection from Individual Activities , 2015, SDM.

[17]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[18]  Emmanuel Bacry,et al.  Uncovering Causality from Multivariate Hawkes Integrated Cumulants , 2016, ICML.

[19]  E. Bacry,et al.  Non-parametric kernel estimation for symmetric Hawkes processes. Application to high frequency financial data , 2011, 1112.1838.

[20]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[21]  Xu Chen,et al.  Benefits from Superposed Hawkes Processes , 2017, AISTATS.

[22]  Le Song,et al.  Learning Social Infectivity in Sparse Low-rank Networks Using Multi-dimensional Hawkes Processes , 2013, AISTATS.

[23]  T. Taimre,et al.  Hawkes Processes , 2015, 1507.02822.

[24]  Jason Eisner,et al.  The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process , 2016, NIPS.

[25]  J. Rasmussen Bayesian Inference for Hawkes Processes , 2013 .

[26]  Hongyuan Zha,et al.  DyRep: Learning Representations over Dynamic Graphs , 2019, ICLR.

[27]  Yu Zhang,et al.  A Survey on Multi-Task Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[28]  Thomas L. Griffiths,et al.  Recasting Gradient-Based Meta-Learning as Hierarchical Bayes , 2018, ICLR.

[29]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[30]  James L. McClelland,et al.  Learning the structure of event sequences. , 1991, Journal of experimental psychology. General.

[31]  A. Hawkes Spectra of some self-exciting and mutually exciting point processes , 1971 .

[32]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[33]  Luc Bauwens,et al.  Département des Sciences Économiques de l'Université catholique de Louvain Modelling Financial High Frequency Data Using Point Processes , 2019 .

[34]  Hongyuan Zha,et al.  Dyadic event attribution in social networks with mixtures of hawkes processes , 2013, CIKM.

[35]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[36]  Lasso and probabilistic inequalities for multivariate point processes , 2015, 1208.0570.

[37]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[38]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[39]  Edoardo M. Airoldi,et al.  Mixed Membership Stochastic Blockmodels , 2007, NIPS.

[40]  Ryan P. Adams,et al.  Gradient-based Hyperparameter Optimization through Reversible Learning , 2015, ICML.

[41]  Alex Beatson,et al.  Amortized Bayesian Meta-Learning , 2018, ICLR.

[42]  P. Reynaud-Bouret,et al.  Adaptive estimation for Hawkes processes; application to genome analysis , 2009, 0903.2919.

[43]  Wenjun Zhang,et al.  Multi-Task Multi-Dimensional Hawkes Processes for Modeling Event Sequences , 2015, IJCAI.

[44]  Boleslaw K. Szymanski,et al.  Overlapping community detection in networks: The state-of-the-art and comparative study , 2011, CSUR.

[45]  Tao Xiang,et al.  Learning to Compare: Relation Network for Few-Shot Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[46]  Sergey Levine,et al.  Probabilistic Model-Agnostic Meta-Learning , 2018, NeurIPS.

[47]  Katherine A. Heller,et al.  Modelling Reciprocating Relationships with Hawkes Processes , 2012, NIPS.

[48]  Le Song,et al.  Multistage Campaigning in Social Networks , 2016, NIPS.

[49]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[50]  Yoshua Bengio,et al.  Learning a synaptic learning rule , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[51]  Mark E. J. Newman,et al.  Stochastic blockmodels and community structure in networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  Jure Leskovec,et al.  SEISMIC: A Self-Exciting Point Process Model for Predicting Tweet Popularity , 2015, KDD.

[53]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[54]  Hongyuan Zha,et al.  Learning Hawkes Processes from Short Doubly-Censored Event Sequences , 2017, ICML.

[55]  Scott W. Linderman,et al.  Discovering Latent Network Structure in Point Process Data , 2014, ICML.