A survey on text generation using generative adversarial networks

Abstract This work presents a thorough review concerning recent studies and text generation advancements using Generative Adversarial Networks. The usage of adversarial learning for text generation is promising as it provides alternatives to generate the so-called “natural” language. Nevertheless, adversarial text generation is not a simple task as its foremost architecture, the Generative Adversarial Networks, were designed to cope with continuous information (image) instead of discrete data (text). Thus, most works are based on three possible options, i.e., Gumbel-Softmax differentiation, Reinforcement Learning, and modified training objectives. All alternatives are reviewed in this survey as they present the most recent approaches for generating text using adversarial-based techniques. The selected works were taken from renowned databases, such as Science Direct, IEEEXplore, Springer, Association for Computing Machinery, and arXiv, whereas each selected work has been critically analyzed and assessed to present its objective, methodology, and experimental results.

[1]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[2]  Andrew M. Dai,et al.  MaskGAN: Better Text Generation via Filling in the ______ , 2018, ICLR.

[3]  Philipp Koehn,et al.  Findings of the 2017 Conference on Machine Translation (WMT17) , 2017, WMT.

[4]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[5]  Jiancheng Lv,et al.  BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation , 2018, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[6]  Yoav Goldberg,et al.  A Primer on Neural Network Models for Natural Language Processing , 2015, J. Artif. Intell. Res..

[7]  Jianmo Ni,et al.  Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects , 2019, EMNLP.

[8]  Shuicheng Yan,et al.  A Survey on Concept Factorization: From Shallow to Deep Representation Learning , 2020, Inf. Process. Manag..

[9]  Mahdieh Soleymani Baghshah,et al.  DGSAN: Discrete Generative Self-Adversarial Network , 2019, Neurocomputing.

[10]  Jing Xiao,et al.  Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators , 2019, AISTATS.

[11]  Zhi Chen,et al.  Adversarial Feature Matching for Text Generation , 2017, ICML.

[12]  Fabio Roli,et al.  Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning , 2017, Pattern Recognit..

[13]  Nina Narodytska,et al.  RelGAN: Relational Generative Adversarial Networks for Text Generation , 2019, ICLR.

[14]  Junli Wang,et al.  Text Generation Service Model Based on Truth-Guided SeqGAN , 2020, IEEE Access.

[15]  Zhe Gan,et al.  Adversarial Text Generation via Feature-Mover's Distance , 2018, NeurIPS.

[16]  Afroz Ahamad,et al.  Generating Text through Adversarial Training Using Skip-Thought Vectors , 2018, NAACL.

[17]  Ping Li,et al.  Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation , 2020, AAAI.

[18]  Robert L. Mercer,et al.  An Estimate of an Upper Bound for the Entropy of English , 1992, CL.

[19]  Jinyin Chen,et al.  Customizable text generation via conditional text generative adversarial network , 2020, Neurocomputing.

[20]  Xuanjing Huang,et al.  Towards Diverse Text Generation with Inverse Reinforcement Learning , 2018, ArXiv.

[21]  Quoc V. Le,et al.  ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators , 2020, ICLR.

[22]  Quan Pan,et al.  A Generative Model for category text generation , 2018, Inf. Sci..

[23]  Yong Yu,et al.  Long Text Generation via Adversarial Training with Leaked Information , 2017, AAAI.

[24]  Linlin Liu,et al.  Collocating Clothes With Generative Adversarial Networks Cosupervised by Categories and Attributes: A Multidiscriminator Framework , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Kevin Lin,et al.  Adversarial Ranking for Language Generation , 2017, NIPS.

[26]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[27]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[28]  Caiquan Xiong,et al.  A Research on Generative Adversarial Networks Applied to Text Generation , 2019, 2019 14th International Conference on Computer Science & Education (ICCSE).

[29]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[30]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[31]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[32]  Giuseppe Rizzo,et al.  Adversarial text generation with context adapted global knowledge and a self-attentive discriminator , 2020, Inf. Process. Manag..

[33]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[34]  Mirella Lapata,et al.  Chinese Poetry Generation with Recurrent Neural Networks , 2014, EMNLP.

[35]  Yang Yang,et al.  FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation , 2020, IEEE Access.

[36]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[37]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[38]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[39]  Yoshua Bengio,et al.  Boundary-Seeking Generative Adversarial Networks , 2017, ICLR 2017.

[40]  Emiel Krahmer,et al.  Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation , 2017, J. Artif. Intell. Res..

[41]  Changde Du,et al.  Semi-supervised cross-modal image generation with generative adversarial networks , 2020, Pattern Recognit..

[42]  Matt J. Kusner,et al.  GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution , 2016, ArXiv.

[43]  Mehdi Rezagholizadeh,et al.  Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation , 2019, NAACL.

[44]  Xu Sun,et al.  Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation , 2018, EMNLP.

[45]  Yang Wang,et al.  Dense Residual Network: Enhancing Global Dense Feature Flow for Character Recognition , 2020 .

[46]  Dale Schuurmans,et al.  Bridging the Gap Between Value and Policy Based Reinforcement Learning , 2017, NIPS.

[47]  Zhenyu Wang,et al.  Emotional Text Generation Based on Cross-Domain Sentiment Transfer , 2019, IEEE Access.

[48]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[49]  Heng Wang,et al.  Text Generation Based on Generative Adversarial Nets with Latent Variable , 2017, PAKDD.

[50]  Yuan Sun,et al.  QuGAN: Quasi Generative Adversarial Network for Tibetan Question Answering Corpus Generation , 2019, IEEE Access.

[51]  I. J. Myung,et al.  Tutorial on maximum likelihood estimation , 2003 .

[52]  Quan Z. Sheng,et al.  Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey , 2019 .

[53]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[54]  Li Wang,et al.  Face hallucination from low quality images using definition-scalable inference , 2019, Pattern Recognit..

[55]  Qingyang Wu,et al.  TextGAIL: Generative Adversarial Imitation Learning for Text Generation , 2020, AAAI.

[56]  Xiaojun Wan,et al.  Automatic generation of sentimental texts via mixture adversarial networks , 2019, Artif. Intell..