Preventing Author Profiling through Zero-Shot Multilingual Back-Translation

Documents as short as a single sentence may inadvertently reveal sensitive information about their authors, including e.g. their gender or ethnicity. Style transfer is an effective way of transforming texts in order to remove any information that enables author profiling. However, for a number of current state-of-theart approaches the improved privacy is accompanied by an undesirable drop in the downstream utility of the transformed data. In this paper, we propose a simple, zero-shot way to effectively lower the risk of author profiling through multilingual back-translation using off-the-shelf translation models. We compare our models with five representative text style transfer models on three datasets across different domains. Results from both an automatic and a human evaluation show that our approach achieves the best overall performance while requiring no training data. We are able to lower the adversarial prediction of gender and race by up to 22% while retaining 95% of the original utility on downstream tasks.

[1]  Lucia Specia,et al.  Personalized Machine Translation: Preserving Original Author Traits , 2016, EACL.

[2]  Lysandre Debut,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[3]  Samuel R. Bowman,et al.  Neural Network Acceptability Judgments , 2018, Transactions of the Association for Computational Linguistics.

[4]  Yoav Goldberg,et al.  Adversarial Removal of Demographic Attributes from Text Data , 2018, EMNLP.

[5]  Lawrence Carin,et al.  Improving Disentangled Text Representation Learning with Information-Theoretic Guidance , 2020, ACL.

[6]  Graham Neubig,et al.  A Probabilistic Formulation of Unsupervised Text Style Transfer , 2020, ICLR.

[7]  Zhiting Hu,et al.  Deep Learning for Text Style Transfer: A Survey , 2020, Computational Linguistics.

[8]  Percy Liang,et al.  Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer , 2018, NAACL.

[9]  Yulia Tsvetkov,et al.  Style Transfer Through Back-Translation , 2018, ACL.

[10]  Guillaume Lample,et al.  Multiple-Attribute Text Rewriting , 2018, ICLR.

[11]  Yiming Yang,et al.  Politeness Transfer: A Tag and Generate Approach , 2020, ACL.

[12]  Mohit Iyyer,et al.  Reformulating Unsupervised Style Transfer as Paraphrase Generation , 2020, EMNLP.

[13]  Yang Zhao,et al.  Unsupervised Rewriter for Multi-Sentence Compression , 2019, ACL.

[14]  Regina Barzilay,et al.  Style Transfer from Non-Parallel Text by Cross-Alignment , 2017, NIPS.

[15]  Kevin Knight,et al.  Obfuscating Gender in Social Media Writing , 2016, NLP+CSS@EMNLP.

[16]  Marcin Junczys-Dowmunt,et al.  The Curious Case of Hallucinations in Neural Machine Translation , 2021, NAACL.

[17]  Florian Schiel,et al.  Multi-Tier Annotations in the Verbmobil Corpus , 2002, LREC.

[18]  Alon Lavie,et al.  METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments , 2005, IEEvaluation@ACL.

[19]  Akhilesh Sudhakar,et al.  “Transforming” Delete, Retrieve, Generate Approach for Controlled Text Style Transfer , 2019, EMNLP.

[20]  Jörg Tiedemann,et al.  Parallel Data, Tools and Interfaces in OPUS , 2012, LREC.

[21]  Yu Cheng,et al.  Domain Adaptive Text Style Transfer , 2019, EMNLP.

[22]  Bernt Schiele,et al.  A4NT: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation , 2017, USENIX Security Symposium.

[23]  Dietrich Klakow,et al.  Privacy Guarantees for De-Identifying Text Transformations , 2020, INTERSPEECH.

[24]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[25]  Lili Mou,et al.  Disentangled Representation Learning for Non-Parallel Text Style Transfer , 2018, ACL.

[26]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[27]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[28]  Dilek Z. Hakkani-Tür,et al.  Preserving Privacy in Spoken Language Databases , 2004 .

[29]  Brendan T. O'Connor,et al.  Demographic Dialectal Variation in Social Media: A Case Study of African-American English , 2016, EMNLP.

[30]  Enhong Chen,et al.  Style Transfer as Unsupervised Machine Translation , 2018, ArXiv.

[31]  Dongyan Zhao,et al.  Style Transfer in Text: Exploration and Evaluation , 2017, AAAI.

[32]  Chenchen Xu,et al.  Privacy-Aware Text Rewriting , 2019, INLG.