Human and Automatic Detection of Generated Text

Recent advancements in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text. The capabilities of humans and automatic discriminators to detect machine-generated text have been a large source of research interest, but humans and machines rely on different cues to make their decisions. Here, we perform careful benchmarking and analysis of three popular sampling-based decoding strategies—top-_k_, nucleus sampling, and untruncated random sampling—and show that improvements in decoding methods have primarily optimized for fooling humans. This comes at the expense of introducing statistical abnormalities that make detection easy for automatic systems. We also show that though both human and automatic detector performance improve with longer excerpt length, even multi-sentence excerpts can fool expert human raters over 30% of the time. Our findings reveal the importance of using both human and automatic detectors to assess the humanness of text generation systems.

[1]  A M Turing,et al.  Computing Machinery and Intelligence A.M. Turing , 2007 .

[2]  Gang Wang,et al.  Serf and turf: crowdturfing for fun and profit , 2011, WWW.

[3]  Jong Kim,et al.  CrowdTarget: Target-based Detection of Crowdturfing in Online Social Networks , 2015, CCS.

[4]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[5]  Georgios Zervas,et al.  Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud , 2015, Manag. Sci..

[6]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[7]  Verena Rieser,et al.  Why We Need New Evaluation Metrics for NLG , 2017, EMNLP.

[8]  Alan Ritter,et al.  Adversarial Learning for Neural Dialogue Generation , 2017, EMNLP.

[9]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[10]  Joelle Pineau,et al.  Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses , 2017, ACL.

[11]  M. Gentzkow,et al.  Social Media and Fake News in the 2016 Election , 2017 .

[12]  Oriol Vinyals,et al.  Adversarial Evaluation of Dialogue Models , 2017, ArXiv.

[13]  Kevin Lin,et al.  Adversarial Ranking for Language Generation , 2017, NIPS.

[14]  Chris J Vargo,et al.  The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016 , 2018, New Media Soc..

[15]  Yann Dauphin,et al.  Hierarchical Neural Story Generation , 2018, ACL.

[16]  Lukasz Kaiser,et al.  Generating Wikipedia by Summarizing Long Sequences , 2018, ICLR.

[17]  Sinan Aral,et al.  The spread of true and false news online , 2018, Science.

[18]  Christopher D. Manning,et al.  Do Massively Pretrained Language Models Make Better Storytellers? , 2019, CoNLL.

[19]  Paul Piwek,et al.  The use of rating and Likert scales in Natural Language Generation human evaluation tasks: A review and some recommendations , 2019, INLG.

[20]  Alexander M. Rush,et al.  GLTR: Statistical Detection and Visualization of Generated Text , 2019, ACL.

[21]  Regina Barzilay,et al.  Are We Safe Yet? The Limitations of Distributional Features for Fake News Detection , 2019, ArXiv.

[22]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[23]  Ali Farhadi,et al.  Defending Against Neural Fake News , 2019, NeurIPS.

[24]  Hany Hassan,et al.  Selecting, Planning, and Rewriting: A Modular Approach for Data-to-Document Generation and Translation , 2019, NGT@EMNLP-IJCNLP.

[25]  Alec Radford,et al.  Release Strategies and the Social Impacts of Language Models , 2019, ArXiv.

[26]  Albert Gatt,et al.  Best practices for the human evaluation of automatically generated text , 2019, INLG.

[27]  H. Womack.  Fake news and alternative facts: information literacy in a post-truth era , 2019, Technical Services Quarterly.

[28]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[29]  Marc'Aurelio Ranzato,et al.  Real or Fake? Learning to Discriminate Machine from Human Generated Text , 2019, ArXiv.

[30]  Hung-Yu Kao,et al.  Probing Neural Network Comprehension of Natural Language Arguments , 2019, ACL.

[31]  Kilian Q. Weinberger,et al.  BERTScore: Evaluating Text Generation with BERT , 2019, ICLR.

[32]  Junichi Yamagishi,et al.  Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection , 2019, AINA.

[33]  Darsh J. Shah,et al.  The Limitations of Stylometry for Detecting Machine-Generated Fake News , 2019, CL.

[34]  S. Kreps,et al.  All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation , 2020, Journal of Experimental Political Science.

[35]  Yejin Choi,et al.  The Curious Case of Neural Text Degeneration , 2019, ICLR.