Multi-Task Neural Models for Translating Between Styles Within and Across Languages

Generating natural language requires conveying content in an appropriate style. We explore two related tasks on generating text of varying formality: monolingual formality transfer and formality-sensitive machine translation. We propose to solve these tasks jointly using multi-task learning, and show that our models achieve state-of-the-art performance for formality transfer and are able to perform formality-sensitive translation without being explicitly trained on style-annotated translation examples.

[1]  Marine Carpuat,et al.  A Study of Style in Machine Translation: Controlling the Formality of Machine Translation Output , 2017, EMNLP.

[2]  Marine Carpuat,et al.  Bi-Directional Neural Machine Translation with Synthetic Parallel Data , 2018, NMT@ACL.

[3]  Graham Neubig,et al.  Extreme Adaptation for Personalized Neural Machine Translation , 2018, ACL.

[4]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[5]  Mamoru Komachi,et al.  Controlling the Voice of a Sentence in Japanese-to-English Neural Machine Translation , 2016, WAT@COLING.

[6]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[7]  Quoc V. Le,et al.  Multi-task Sequence to Sequence Learning , 2015, ICLR.

[8]  Ralph Grishman,et al.  Paraphrasing for Style , 2012, COLING.

[9]  Hideki Mima,et al.  Improving Performance of Transfer-Driven Machine Translation with Extra-Linguistic Informatioon from Context, Situation and Environment , 1997, IJCAI.

[10]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[11]  Alexander M. Rush,et al.  OpenNMT: Open-Source Toolkit for Neural Machine Translation , 2017, ACL.

[12]  Eric P. Xing,et al.  Toward Controlled Generation of Text , 2017, ICML.

[13]  Chris Callison-Burch,et al.  Optimizing Statistical Machine Translation for Text Simplification , 2016, TACL.

[14]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[15]  Douglas Biber,et al.  Using multi-dimensional analysis to explore cross-linguistic universals of register variation , 2014 .

[16]  Rich Caruana,et al.  Multitask Learning , 1997, Machine-mediated learning.

[17]  Martin Wattenberg,et al.  Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation , 2016, TACL.

[18]  Ping Chen,et al.  Text Simplification Using Neural Machine Translation , 2016, AAAI.

[19]  Philipp Koehn,et al.  Statistical Significance Tests for Machine Translation Evaluation , 2004, EMNLP.

[20]  Joel R. Tetreault,et al.  Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer , 2018, NAACL.

[21]  Xiaodong Liu,et al.  Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval , 2015, NAACL.

[22]  Matt Post,et al.  We start by defining the recurrent architecture as implemented in S OCKEYE , following , 2018 .

[23]  Christian Federmann,et al.  Applying Cross-Entropy Difference for Selecting Parallel Training Data from Publicly Available Sources for Conversational Machine Translation , 2015 .

[24]  Regina Barzilay,et al.  Style Transfer from Non-Parallel Text by Cross-Alignment , 2017, NIPS.

[25]  Marine Carpuat,et al.  Discovering Stylistic Variations in Distributional Vector Space Models via Lexical Paraphrases , 2017 .

[26]  Diana Inkpen,et al.  Generation of Formal and Informal Sentences , 2011, ENLG.

[27]  David Kauchak,et al.  Learning to Simplify Sentences Using Wikipedia , 2011, Monolingual@ACL.

[28]  Rico Sennrich,et al.  Controlling Politeness in Neural Machine Translation via Side Constraints , 2016, NAACL.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  William D. Lewis,et al.  Intelligent Selection of Language Model Training Data , 2010, ACL.

[31]  Tommi S. Jaakkola,et al.  Sequence to Better Sequence: Continuous Revision of Combinatorial Structures , 2017, ICML.

[32]  Marine Carpuat,et al.  Identifying Semantic Divergences in Parallel Text without Annotations , 2018, NAACL.

[33]  Lucia Specia,et al.  Personalized Machine Translation: Preserving Original Author Traits , 2016, EACL.

[34]  Emiel Krahmer,et al.  Sentence Simplification by Monolingual Machine Translation , 2012, ACL.

[35]  Marine Carpuat,et al.  The UMD Machine Translation Systems at IWSLT 2016: English-to-French Translation of Speech Transcripts , 2016, IWSLT.

[36]  Jörg Tiedemann,et al.  OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles , 2016, LREC.

[37]  Dongyan Zhao,et al.  Style Transfer in Text: Exploration and Evaluation , 2017, AAAI.

[38]  Jean-Marc Dewaele,et al.  Formality of Language: definition, measurement and behavioral determinants , 1999 .

[39]  Chenhui Chu,et al.  An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation , 2017, ACL.

[40]  Eduard Hovy,et al.  Generating Natural Language Under Pragmatic Constraints , 1988 .

[41]  Lior Wolf,et al.  Using the Output Embedding to Improve Language Models , 2016, EACL.

[42]  Yulia Tsvetkov,et al.  Style Transfer Through Back-Translation , 2018, ACL.

[43]  Mirella Lapata,et al.  Sentence Simplification with Deep Reinforcement Learning , 2017, EMNLP.

[44]  Iryna Gurevych,et al.  A Monolingual Tree-based Translation Model for Sentence Simplification , 2010, COLING.

[45]  Josep Maria Crego,et al.  Domain Control for Neural Machine Translation , 2016, RANLP.

[46]  Harsh Jhamtani,et al.  Shakespearizing Modern Language Using Copy-Enriched Sequence to Sequence Models , 2017, Proceedings of the Workshop on Stylistic Variation.