Generating Text via Adversarial Training

Generative Adversarial Networks (GANs) have achieved great success in generating realistic synthetic real-valued data. However, the discrete output of language model hinders the application of gradient-based GANs. In this paper we propose a generic framework employing Long short-term Memory (LSTM) and convolutional neural network (CNN) for adversarial training to generate realistic text. Instead of using standard objective of GAN, we match the feature distribution when training the generator. In addition, we use various techniques to pre-train the model and handle discrete intermediate variables. We demonstrate that our model can generate realistic sentence using adversarial training.