Sentence Simplification from Non-Parallel Corpus with Adversarial Learning

In this study, we propose sentence simplification from a non-parallel corpus with adversarial learning. In recent years, sentence simplification based on a statistical machine translation framework and neural networks have been actively studied. However, most methods require a large parallel corpus, which is expensive to build. In this paper, our purpose is sentence simplification with a non-parallel corpus in open data en-Wikipedia and Simple-Wikipedia articles. We use a style transfer framework with adversarial learning for learning by non-parallel corpus and adapted a prior work [by Barzilay et al.] to sentence simplification as a base framework. Furthermore, from the perspective of improving retention of sentence meaning, we add pretraining reconstruction loss and cycle consistency loss to the base framework. We also improve the sentence quality output from the proposed model as a result of the expansion. CCS CONCEPTS • Computing methodologies $\rightarrow$ Natural language generation.

[1]  Regina Barzilay,et al.  Style Transfer from Non-Parallel Text by Cross-Alignment , 2017, NIPS.

[2]  Raman Chandrasekar,et al.  Automatic induction of rules for text simplification , 1997, Knowl. Based Syst..

[3]  Sergiu Nisioi,et al.  A Detailed Evaluation of Neural Sequence-to-Sequence Models for In-domain and Cross-domain Text Simplification , 2018, LREC.

[4]  Eric P. Xing,et al.  Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2014, ACL 2014.

[5]  Yulia Tsvetkov,et al.  Style Transfer Through Back-Translation , 2018, ACL.

[6]  Cristian Danescu-Niculescu-Mizil,et al.  For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia , 2010, NAACL.

[7]  Iryna Gurevych,et al.  A Monolingual Tree-based Translation Model for Sentence Simplification , 2010, COLING.

[8]  David Kauchak,et al.  Improving Text Simplification Language Modeling Using Unsimplified Text Data , 2013, ACL.

[9]  Eneko Agirre,et al.  Unsupervised Statistical Machine Translation , 2018, EMNLP.

[10]  Hong Yu,et al.  Neural Semantic Encoders , 2016, EACL.

[11]  Raman Chandrasekar,et al.  Motivations and Methods for Text Simplification , 1996, COLING.

[12]  Andrew M. Dai,et al.  MaskGAN: Better Text Generation via Filling in the ______ , 2018, ICLR.

[13]  Hong Yu,et al.  Sentence Simplification with Memory-Augmented Neural Networks , 2018, NAACL.

[14]  Michael Strube,et al.  Dependency Tree Based Sentence Compression , 2008, INLG.

[15]  Jason Weston,et al.  A Neural Attention Model for Abstractive Sentence Summarization , 2015, EMNLP.

[16]  Chris Callison-Burch,et al.  Optimizing Statistical Machine Translation for Text Simplification , 2016, TACL.

[17]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[18]  Eric P. Xing,et al.  Toward Controlled Generation of Text , 2017, ICML.

[19]  Ming-Yu Liu,et al.  Coupled Generative Adversarial Networks , 2016, NIPS.

[20]  Harsh Jhamtani,et al.  Shakespearizing Modern Language Using Copy-Enriched Sequence to Sequence Models , 2017, Proceedings of the Workshop on Stylistic Variation.

[21]  R. Flesch.  A new readability yardstick. , 1948, The Journal of applied psychology.

[22]  Emiel Krahmer,et al.  Sentence Simplification by Monolingual Machine Translation , 2012, ACL.

[23]  Percy Liang,et al.  Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer , 2018, NAACL.

[24]  Lucia Specia,et al.  Supporting the Adaptation of Texts for Poor Literacy Readers: a Text Simplification Editor for Brazilian Portuguese , 2009, BEA@NAACL.

[25]  Zhen Xu,et al.  Neural Response Generation via GAN with an Approximate Embedding Layer , 2017, EMNLP.

[26]  Shashi Narayan,et al.  Hybrid Simplification using Deep Semantics and Machine Translation , 2014, ACL.

[27]  Yang Liu,et al.  Towards Robust Neural Machine Translation , 2018, ACL.

[28]  Eric P. Xing,et al.  Unsupervised Text Style Transfer using Language Models as Discriminators , 2018, NeurIPS.

[29]  Daniel Marcu,et al.  Text Simplification for Information-Seeking Applications , 2004, CoopIS/DOA/ODBASE.

[30]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[31]  Wei Chen,et al.  Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets , 2017, NAACL.

[32]  Eneko Agirre,et al.  Unsupervised Neural Machine Translation , 2017, ICLR.

[33]  Lucia Specia,et al.  Unsupervised Lexical Simplification for Non-Native Speakers , 2016, AAAI.

[34]  Satoshi Sato,et al.  Verb Paraphrase based on Case Frame Alignment , 2002, ACL.

[35]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[36]  Mirella Lapata,et al.  Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming , 2011, EMNLP.

[37]  Noémie Elhadad,et al.  Putting it Simply: a Context-Aware Approach to Lexical Simplification , 2011, ACL.

[38]  Siobhan Devlin,et al.  Simplifying Text for Language-Impaired Readers , 1999, EACL.

[39]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[40]  Mirella Lapata,et al.  Sentence Simplification with Deep Reinforcement Learning , 2017, EMNLP.

[41]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[42]  Kentaro Inui,et al.  Text Simplification for Reading Assistance: A Project Note , 2003, IWP@ACL.

[43]  Zhi Chen,et al.  Adversarial Feature Matching for Text Generation , 2017, ICML.

[44]  Ping Chen,et al.  Text Simplification Using Neural Machine Translation , 2016, AAAI.

[45]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[46]  Alan Ritter,et al.  Adversarial Learning for Neural Dialogue Generation , 2017, EMNLP.

[47]  Goran Glavas,et al.  Simplifying Lexical Simplification: Do We Need Simplified Corpora? , 2015, ACL.

[48]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).