Examining the Roles of Automation, Crowds and Professionals Towards Sustainable Fact-checking

This study explores an online fact-checking community called politicalfactchecking on reddit.com that relies on crowdsourcing to find and verify check-worthy facts relating to U.S. politics. The community embodies a network journalism model in which the process of finding and verifying check-worthy facts through crowdsourcing is coordinated by a team of moderators. Applying the concepts of connective journalism, this study analyzed the posts (N = 543) and comments (N = 10, 221) on the community’s Reddit page to understand differences in roles of the community members and the moderators. A mixed-method approach was used to analyze the data. The authors also developed an automated argument classification model to analyze the contents and identify ways to automate parts of the process. The findings suggest that a model consisting of crowds, professionals, and computer-assisted analysis could increase efficiency and decrease costs in news organizations that involve fact-checking.

[1]  David L. Altheide.  Qualitative Media Analysis , 1996 .

[2]  Naeemul Hassan,et al.  The Quest to Automate Fact-Checking , 2015 .

[3]  Chengkai Li,et al.  Toward Automated Fact-Checking: Detecting Check-worthy Factual Claims by ClaimBuster , 2017, KDD.

[4]  Cong Yu,et al.  Computational Journalism: A Call to Arms to Database Researchers , 2011, CIDR.

[5]  Saif Mohammad,et al.  A Dataset for Detecting Stance in Tweets , 2016, LREC.

[6]  S. Hayakawa.  Language in Thought and Action , 1949 .

[7]  Naeemul Hassan,et al.  Comparing Automated Factual Claim Detection Against Judgments of Journalism Organizations , 2016 .

[8]  Mahmoud Fouz,et al.  Why rumors spread so quickly in social networks , 2012, Commun. ACM.

[9]  Brant Houston.  The future of investigative journalism , 2010, Daedalus.

[10]  Chengkai Li,et al.  Detecting Check-worthy Factual Claims in Presidential Debates , 2015, CIKM.

[11]  Sarah Cohen,et al.  Computational journalism , 2011, Commun. ACM.

[12]  Lucas Graves,et al.  Deciding What's True: Fact-Checking Journalism and the New Ecology of News , 2013 .

[13]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[14]  Parinaz Sobhani.  Stance Detection and Analysis in Social Media , 2017 .

[15]  Brett Walenz,et al.  Finding , Monitoring , and Checking Claims Computationally Based on Structured Data The iCheck / uClaim Team , 2014 .

[16]  Chengkai Li,et al.  ClaimBuster: The First-ever End-to-end Fact-checking System , 2017, Proc. VLDB Endow..