TextDream: Conditional Text Generation by Searching in the Semantic Space

Conditional text generation is a fundamental task in natural language generation. Traditional conditional generative models build conditional probability distributions over the given labels. However, categorical label information is usually very abstract, e.g., sentiment, and it is difficult to be disentangled from the content. Therefore, instead of generating text by modeling conditional probability distribution, we propose a novel text generation method TextDream through searching in the semantic space. Specifically, in this method, a random text seed is initially given and the new text is generated by local search operation. The generation procedure is guided by a fitness function, typically a classification model. Text with higher fitness will be preserved. This procedure loops until the qualified solution is found. Experimental results show that our method is able to generate more diverse text compared with advanced conditional generative models.

[1]  Ole Winther,et al.  How to Train Deep Variational Autoencoders and Probabilistic Ladder Networks , 2016, ICML 2016.

[2]  Ying Tan,et al.  Fireworks Algorithm for Optimization , 2010, ICSI.

[3]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[4]  Zhiting Hu,et al.  Improved Variational Autoencoders for Text Modeling using Dilated Convolutions , 2017, ICML.

[5]  Honglak Lee,et al.  Attribute2Image: Conditional Image Generation from Visual Attributes , 2015, ECCV.

[6]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[7]  Ole Winther,et al.  Auxiliary Deep Generative Models , 2016, ICML.

[8]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[9]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[10]  Ruslan Salakhutdinov,et al.  Generating Images from Captions with Attention , 2015, ICLR.

[11]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[12]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[13]  Fei-Fei Li,et al.  Deep visual-semantic alignments for generating image descriptions , 2015, CVPR.

[14]  Sanja Fidler,et al.  Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  Ke Ding,et al.  Introduction to Fireworks Algorithm , 2013, Int. J. Swarm Intell. Res..

[17]  Yoshua Bengio,et al.  Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[20]  Ying Tan,et al.  Variational Autoencoder for Semi-Supervised Text Classification , 2017, AAAI.

[21]  Ying Tan,et al.  A Cooperative Framework for Fireworks Algorithm , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[22]  Chong Wang,et al.  TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency , 2016, ICLR.

[23]  Eric P. Xing,et al.  Controllable Text Generation , 2017, ArXiv.

[24]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[25]  David E. Goldberg,et al.  Dynamic System Control Using Rule Learning and Genetic Algorithms , 1985, IJCAI.

[26]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[27]  Ying Tan,et al.  Fireworks algorithm with differential mutation for solving the CEC 2014 competition problems , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[28]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[29]  Jianhua Liu,et al.  The Improvement on Controlling Exploration and Exploitation of Firework Algorithm , 2013, ICSI.