One Source, Two Targets: Challenges and Rewards of Dual Decoding

Machine translation is generally understood as generating one target text from an input source document. In this paper, we consider a stronger requirement: to jointly generate two texts so that each output side effectively depends on the other. As we discuss, such a device serves several practical purposes, from multi-target machine translation to the generation of controlled variations of the target text. We present an analysis of possible implementations of dual decoding, and experiment with four applications. Viewing the problem from multiple angles allows us to better highlight the challenges of dual decoding and to also thoroughly analyze the benefits of generating matched, rather than independent, translations.

[1]  Yue Zhang,et al.  Code-Switching for Enhancing NMT with Pre-Specified Translation , 2019, NAACL.

[2]  Mauro Cettolo,et al.  WIT3: Web Inventory of Transcribed and Translated Talks , 2012, EAMT.

[3]  Chenhui Chu,et al.  A Survey of Multilingual Neural Machine Translation , 2019, ACM Comput. Surv..

[4]  Kamel Smaïli,et al.  Machine Translation on a Parallel Code-Switched Corpus , 2019, Canadian AI.

[5]  Jiajun Zhang,et al.  Synchronous Bidirectional Neural Machine Translation , 2019, TACL.

[6]  Lior Wolf,et al.  Using the Output Embedding to Improve Language Models , 2016, EACL.

[7]  Jiajun Zhang,et al.  Synchronously Generating Two Languages with Interactive Decoding , 2019, EMNLP.

[8]  Kevin Duh,et al.  Multi-Target Machine Translation with Multi-Synchronous Context-free Grammars , 2015, NAACL.

[9]  Marine Carpuat,et al.  Mixed Language and Code-Switching in the Canadian Hansard , 2014, CodeSwitch@EMNLP.

[10]  Yang Liu,et al.  Exploiting reverse target-side contexts for neural machine translation via asynchronous bidirectional decoding , 2019, Artif. Intell..

[11]  Myle Ott,et al.  fairseq: A Fast, Extensible Toolkit for Sequence Modeling , 2019, NAACL.

[12]  Jakob Verbeek,et al.  Efficient Wait-k Models for Simultaneous Machine Translation , 2020, INTERSPEECH.

[13]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[14]  Marco Gaido,et al.  Between Flexibility and Consistency: Joint Generation of Captions and Subtitles , 2021, IWSLT.

[15]  Taro Watanabe,et al.  Bidirectional Decoding for Statistical Machine Translation , 2002, COLING.

[16]  Jiajun Zhang,et al.  Synchronous Bidirectional Inference for Neural Sequence Generation , 2019, Artif. Intell..

[17]  André F. T. Martins,et al.  Scheduled Sampling for Transformers , 2019, ACL.

[18]  Dianhai Yu,et al.  Multi-Task Learning for Multiple Language Translation , 2015, ACL.

[19]  Rico Sennrich,et al.  Controlling Politeness in Neural Machine Translation via Side Constraints , 2016, NAACL.

[20]  Tanmoy Chakraborty,et al.  SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets , 2020, SEMEVAL.

[21]  Pascale Fung,et al.  Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences , 2019, CoNLL.

[22]  Rongrong Ji,et al.  Asynchronous Bidirectional Decoding for Neural Machine Translation , 2018, AAAI.

[23]  Hakan Inan,et al.  Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling , 2016, ICLR.

[24]  Laura Kallmeyer,et al.  Multilingual Code-switching Identification via LSTM Recurrent Neural Networks , 2016, CodeSwitch@EMNLP.

[25]  Julia Hirschberg,et al.  Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task , 2018, CodeSwitch@ACL.

[26]  Hermann Ney,et al.  Statistical multi-source translation , 2001, MTSUMMIT.

[27]  Noah A. Smith,et al.  A Simple, Fast, and Effective Reparameterization of IBM Model 2 , 2013, NAACL.

[28]  Ryan Cotterell,et al.  Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology , 2019, ACL.

[29]  Samy Bengio,et al.  Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks , 2015, NIPS.

[30]  Franccois Yvon,et al.  Can You Traducir This? Machine Translation for Code-Switched Input , 2021, CALCS.

[31]  Yoshua Bengio,et al.  Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism , 2016, NAACL.

[32]  Chengqing Zong,et al.  Synchronous Interactive Decoding for Multilingual Neural Machine Translation , 2021, AAAI.

[33]  Matt Post,et al.  A Call for Clarity in Reporting BLEU Scores , 2018, WMT.

[34]  Lane Schwartz,et al.  Multi-Source Translation Methods , 2008, AMTA.

[35]  Dan Garrette,et al.  Part-of-Speech Tagging for Code-Switched, Transliterated Texts without Explicit Language Identification , 2018, EMNLP.

[36]  Marine Carpuat,et al.  Controlling Neural Machine Translation Formality with Synthetic Supervision , 2019, AAAI.

[37]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[38]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[39]  Jan Niehues,et al.  Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder , 2016, IWSLT.

[40]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[41]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[42]  David Chiang,et al.  Tied Multitask Learning for Neural Speech Translation , 2018, NAACL.

[43]  Yan Song,et al.  Meet Changes with Constancy: Learning Invariance in Multi-Source Translation , 2020, COLING.

[44]  Alan W. Black,et al.  A Survey of Code-switched Speech and Language Processing , 2019, ArXiv.

[45]  Eiichiro Sumita,et al.  Bidirectional Phrase-based Statistical Machine Translation , 2009, EMNLP.

[46]  Kevin Knight,et al.  Multi-Source Neural Translation , 2016, NAACL.

[47]  A. Gispert,et al.  Reordered Search, and Tuple Unfolding for Ngram-based SMT , 2005, MTSUMMIT.

[48]  Feifei Zhai,et al.  Three Strategies to Improve One-to-Many Multilingual Translation , 2018, EMNLP.

[49]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[50]  Martin Wattenberg,et al.  Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation , 2016, TACL.