Readers’ perception of computer-generated news: Credibility, expertise, and readability

We conducted an online experiment to study people’s perception of automated computer-written news. Using a 2 × 2 × 2 design, we varied the article topic (sports, finance; within-subjects) and both the articles’ actual and declared source (human-written, computer-written; between-subjects). Nine hundred eighty-six subjects rated two articles on credibility, readability, and journalistic expertise. Varying the declared source had small but consistent effects: subjects rated articles declared as human written always more favorably, regardless of the actual source. Varying the actual source had larger effects: subjects rated computer-written articles as more credible and higher in journalistic expertise but less readable. Across topics, subjects’ perceptions did not differ. The results provide conservative estimates for the favorability of computer-written news, which will further increase over time and endorse prior calls for establishing ethics of computer-written news.

[1]  A. A. Lumsdaine.  Communication and persuasion , 1954 .

[2]  H. Kelley,et al.  Communication and Persuasion: Psychological Studies of Opinion Change , 1982 .

[3]  Philip Meyer,et al.  Defining and Measuring Credibility of Newspapers: Developing an Index , 1988 .

[4]  Mark D. West,et al.  Validating a Scale for the Measurement of Credibility: A Covariance Structure Modeling Approach , 1994 .

[5]  Alison A. Plessinger,et al.  Exploring Receivers' Criteria for Perception of Print and Online News , 1999 .

[6]  Thomas J. Johnson,et al.  Wag the Blog: How Reliance on Traditional Media and the Internet Influence Credibility Perceptions of Weblogs Among Blog Users , 2004 .

[7]  Miriam J. Metzger,et al.  Digital Media and Youth: Unparalleled Opportunity and Unprecedented Responsibility , 2008 .

[8]  Miriam J. Metzger,et al.  Social and Heuristic Approaches to Credibility Evaluation Online , 2010 .

[9]  Thomas J. Johnson,et al.  New Perspectives on Media Credibility Research , 2010 .

[10]  Sarah Cohen,et al.  Computational journalism , 2011, Commun. ACM.

[11]  Eli Pariser,et al.  The Filter Bubble: What the Internet Is Hiding from You , 2011 .

[12]  Arjen van Dalen,et al.  The algorithms behind the headlines. How machine-written news redefines the core skills of human journalists , 2012 .

[13]  David Matthews,et al.  Unsupervised joke generation from big data , 2013, ACL.

[14]  Cw Anderson,et al.  Towards a sociology of computational and algorithmic journalism , 2013, New Media Soc..

[15]  D. Lazer,et al.  The Parable of Google Flu: Traps in Big Data Analysis , 2014, Science.

[16]  Brandon Van Der Heide,et al.  Social Media as Information Source: Recency of Updates and Credibility of Information , 2014, J. Comput. Mediat. Commun..

[17]  Wolfgang Schweiger,et al.  News Quality from the Recipients' Perspective , 2014 .

[18]  E. Krahmer,et al.  Journalist versus news consumer : The perceived credibility of machine written news , 2014 .

[19]  Christer Clerwall.  Enter the Robot Journalist , 2014 .

[20]  Philip M. Napoli.  Automated Media: An Institutional Theory Perspective on Algorithmic Media Production and Consumption , 2014 .

[21]  Alfred Hermida,et al.  From Mr. and Mrs. Outlier To Central Tendencies , 2015 .

[22]  S. Lewis,et al.  Actors, Actants, Audiences, and Activities in Cross-Media News Work , 2015 .

[23]  Alexander Winkler,et al.  Implicit Force Control of a Position Controlled Robot – A Comparison with Explicit Algorithms , 2015 .

[24]  M. Carlson,et al.  The Robotic Reporter , 2015 .

[25]  Bastian Haarmann,et al.  Natural Language News Generation from Big Data , 2015 .

[26]  Noam Lemelshtrich Latar.  The Robot Journalist in the Age of Social Physics: The End of Human Journalism? , 2015 .

[27]  Nicholas Diakopoulos,et al.  Algorithmic Accountability , 2015 .

[28]  Amílcar Cardoso,et al.  Poetry Generation with PoeTryMe , 2015 .

[29]  Andreas Graefe,et al.  Guide to automated journalism , 2016 .