A topic-driven language model for learning to generate diverse sentences

Abstract Generating diverse sentences under a topic is a meaningful, yet not well-solved task in the field of natural language processing. We present a neural language model for generating diverse sentences conditioned on a given topic distribution. From the perspective of diversity, the proposed model takes the advantages of variational autoencoders with convolutional neural network and long short-term memory architecture. The proposed model is trained end-to-end to learn topic-level Gaussian distributions in the latent space. Then our model decodes the samples of topics obtained from latent space to generate each sentence. Results on Restaurant Dataset and Yahoo! Answers Dataset show that our model outperforms other methods in terms of language model perplexity. Also, our approach can generate a large set of different coherent sentences related to given topics. The diversity of our sentences provides a novel interpretation of topics.