A Method for Constructing Supervised Time Topic Model Based on Variational Autoencoder

Topic modeling is a probabilistic generation model to find the representative topic of a document and has been successfully applied to various document-related tasks in recent years. Especially in the supervised topic model and time topic model, many methods have achieved some success. The supervised topic model can learn topics from documents annotated with multiple labels and the time topic model can learn topics that evolve over time in a sequentially organized corpus. However, there are some documents with multiple labels and time-stamped in reality, which need to construct a supervised time topic model to achieve document-related tasks. There are few research papers on the supervised time topic model. To solve this problem, we propose a method for constructing a supervised time topic model. By analysing the generative process of the supervised topic model and time topic model, respectively, we introduce the construction process of the supervised time topic model based on variational autoencoder in detail and conduct preliminary experiments. Experimental results demonstrate that the supervised time topic model outperforms several state-of-the-art topic models.

[1]  Dongwoo Kim,et al.  Dirichlet Process with Mixed Random Measures: A Nonparametric Topic Model for Labeled Data , 2012, ICML.

[2]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[3]  Ramesh Nallapati,et al.  Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora , 2009, EMNLP.

[4]  Kenli Li,et al.  An Improved LDA Multi-document Summarization Model Based on TensorFlow , 2017, 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI).

[5]  Zheng Huo,et al.  A Method for Constructing Supervised Topic Model Based on Term Frequency-Inverse Topic Frequency , 2019, Symmetry.

[6]  Susan T. Dumais,et al.  Partially labeled topic models for interpretable text mining , 2011, KDD.

[7]  Charles A. Sutton,et al.  Autoencoding Variational Inference For Topic Models , 2017, ICLR.

[8]  Il-Chul Moon,et al.  Supervised Dynamic Topic Models for Associative Topic Extraction with A Numerical Time Series , 2015, TM@CIKM.

[9]  Michael I. Jordan,et al.  DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification , 2008, NIPS.

[10]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.

[11]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[12]  Timothy Baldwin,et al.  Automatic Evaluation of Topic Coherence , 2010, NAACL.

[13]  David Kauchak,et al.  Modeling word burstiness using the Dirichlet distribution , 2005, ICML.

[14]  Marc'Aurelio Ranzato,et al.  Dynamic auto-encoders for semantic indexing , 2010 .

[15]  Charles Elkan,et al.  Accounting for burstiness in topic models , 2009, ICML '09.

[16]  Elaine Zosa,et al.  Multilingual Dynamic Topic Model , 2019, RANLP.

[17]  Timothy Baldwin,et al.  Machine Reading Tea Leaves: Automatically Evaluating Topic Coherence and Topic Model Quality , 2014, EACL.

[18]  David M. Blei,et al.  Supervised Topic Models , 2007, NIPS.