Towards unsupervised text multi-style transfer with parameter-sharing scheme

Abstract Text style transfer is an important task in the field of natural language generation. Because of the lack of parallel data, it is a challenge to address this problem in an unsupervised manner. Existing methods mainly focus on the two-style transfer task, i.e. from one source style to one target style. In this paper, we first propose the task of unsupervised text multi-style transfer to address the problem of efficient text transfer from a source style to multiple target styles. To tackle this new task, we present a novel model based on Non-Autoregressive Transformer (NAT). The model consists of two parts: a parameter-shared style-independent module and a style-dependent module. In practice, we only need to reinitialize the parameter of style-dependent modules and retrain the whole model which can converge fast. Experimental results show that our model not only performs well in two-style transfer task, but also promises good results in the multi-style scenario.

[1]  Regina Barzilay,et al.  Style Transfer from Non-Parallel Text by Cross-Alignment , 2017, NIPS.

[2]  Kevin Knight,et al.  Large Scale Decipherment for Out-of-Domain Machine Translation , 2012, EMNLP-CoNLL.

[3]  Jonathon Shlens,et al.  A Learned Representation For Artistic Style , 2016, ICLR.

[4]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[5]  Tsung-Hsien Wen,et al.  Latent Intention Dialogue Models , 2017, ICML.

[6]  Percy Liang,et al.  Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer , 2018, NAACL.

[7]  Dongyan Zhao,et al.  Style Transfer in Text: Exploration and Evaluation , 2017, AAAI.

[8]  Kevin Knight,et al.  Deciphering Related Languages , 2017, EMNLP.

[9]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[10]  Xuanjing Huang,et al.  Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation , 2019, ACL.

[11]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[12]  Yulia Tsvetkov,et al.  Style Transfer Through Back-Translation , 2018, ACL.

[13]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[14]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[15]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[16]  Eric P. Xing,et al.  Toward Controlled Generation of Text , 2017, ICML.

[17]  Samy Bengio,et al.  Content preserving text generation with attribute controls , 2018, NeurIPS.

[18]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[19]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Guillaume Lample,et al.  Multiple-Attribute Text Rewriting , 2018, ICLR.

[23]  Victor O. K. Li,et al.  Non-Autoregressive Neural Machine Translation , 2017, ICLR.

[24]  Eric P. Xing,et al.  Unsupervised Text Style Transfer using Language Models as Discriminators , 2018, NeurIPS.

[25]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[26]  Lior Wolf,et al.  Unsupervised Cross-Domain Image Generation , 2016, ICLR.

[27]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[29]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.