An experimental study of the effectiveness of fact checks: interplay of evidence type, veracity and news agreement

PurposeGiven the rapid growth in efforts on misinformation correction, the study aims to test how evidence type and veracity interact with news agreement on the effectiveness of fact-checking on how well a corrective message discount a false news information.Design/methodology/approachExperimental participants (N = 511) in Hong Kong were exposed to the same news article and then to a piece of corrective information debunking the news article with variation in the types of evidence (numerical vs narrative) and veracity (no verdict vs half false vs entirely false) in 2019.FindingsAmong the participants who disagreed with the news article, numerical fact-checking was more effective than narrative fact-checking in discounting the news article. Some evidence of the backfire effect was found among participants for whom the article was attitude incongruent.Originality/valueWhen debunking false information with people exposed to attitude-incongruent news, a milder verdict presented in the form of a half-false scale can prompt a more positive perception of the issue at stake than an entirely false scale, implying that a less certain verdict can help in mitigating the backfire effect compared to a certain verdict.

[1]  S. Tsang,et al.  How People Process Different Types of Health Misinformation: Roles of Content Falsity and Evidence Type. , 2023, Health communication.

[2]  S. Kim,et al.  "Masks do not work": COVID-19 misperceptions and theory-driven corrective strategies on Facebook , 2022, Online Inf. Rev..

[3]  S. Tsang.  Biased, not lazy: assessing the effect of COVID-19 misinformation tactics on perceptions of inaccuracy and fakeness , 2022, Online Media and Global Communication.

[4]  Yonghwan Kim,et al.  Thinking, checking and learning: testing a moderated-mediation model of social media news use conditional upon elaboration on political knowledge via fact-checking , 2021, Online Inf. Rev..

[5]  Luke Kien-Weng Tan,et al.  Exploring how online responses change in response to debunking messages about COVID-19 on WhatsApp , 2021, Online Inf. Rev..

[6]  Jonathan A. Busam,et al.  Real Solutions for Fake News? Measuring the Effectiveness of General Warnings and Fact-Check Tags in Reducing Belief in False Stories on Social Media , 2020, Political Behavior.

[7]  S. Tsang.  Motivated Fake News Perception: The Impact of News Sources and Policy Support on Audiences’ Assessment of News Fakeness , 2020, Journalism & Mass Communication Quarterly.

[8]  S. Tsang.  Issue stance and perceived journalistic motives explain divergent audience perceptions of fake news , 2020, Journalism.

[9]  Jonathan Cohen,et al.  Fact-Checking: A Meta-Analysis of What Works and for Whom , 2019, Political Communication.

[10]  Jakob Henke,et al.  How can Journalists Promote News Credibility? Effects of Evidences on Trust and Credibility , 2020, Journalism Practice.

[11]  Nathan Walter,et al.  A Meta-Analytic Examination of the Continued Influence of Misinformation in the Face of Correction: How Powerful Is It, Why Does It Happen, and How to Stop It? , 2020, Communication Research.

[12]  David G. Rand,et al.  Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning , 2019, Cognition.

[13]  Ethan Porter,et al.  The Elusive Backfire Effect: Mass Attitudes’ Steadfast Factual Adherence , 2019 .

[14]  P. Mena.  Principles and Boundaries of Fact-checking: Journalists’ Perceptions , 2018, Journalism Practice.

[15]  Asbjørn Følstad,et al.  How Journalists and Social Media Users Perceive Online Fact-Checking and Verification Services , 2018 .

[16]  Nathan Walter,et al.  How to unring the bell: A meta-analytic approach to correction of misinformation , 2018 .

[17]  Ashley Muddiman,et al.  Correcting Political and Consumer Misperceptions: The Effectiveness and Effects of Rating Scale Versus Contextual Correction Formats , 2018 .

[18]  K. Jamieson,et al.  Fact-Checking Effectiveness as a Function of Format and Tone: Evaluating FactCheck.org and FlackCheck.org , 2018 .

[19]  Ethan Porter,et al.  Sex Trafficking, Russian Infiltration, Birth Certificates, and Pedophilia: A Survey Experiment Correcting Fake News , 2018 .

[20]  Kjerstin Thorson,et al.  Partisan Selective Sharing: The Biased Diffusion of Fact-Checking Messages on Social Media , 2017 .

[21]  Lucas Graves,et al.  The rise of fact-checking sites in Europe , 2016 .

[22]  Lucas Graves,et al.  Boundaries Not Drawn , 2016 .

[23]  Brendan Nyhan,et al.  Understanding Innovations in Journalistic Practice: A Field Experiment Examining Motivations for Fact‐Checking , 2016 .

[24]  Kim L. Fridkin,et al.  Liar, Liar, Pants on Fire: How Fact-Checking Influences Citizens’ Reactions to Negative Advertising , 2015 .

[25]  T. Levine,et al.  Comparing Separate Process and Intertwined Models for Reactance , 2013 .

[26]  R. Kelly Garrett,et al.  The promise and peril of real-time corrections to political misperceptions , 2013, CSCW.

[27]  Lourdes Baezconde-Garbanati,et al.  Narrative versus Non-narrative: The Role of Identification, Transportation and Emotion in Reducing Health Disparities. , 2013, The Journal of communication.

[28]  Ullrich K. H. Ecker,et al.  Misinformation and Its Correction , 2012, Psychological science in the public interest : a journal of the American Psychological Society.

[29]  Bing Han,et al.  How Do Statistical and Narrative Evidence Affect Persuasion?: The Role of Evidentiary Features , 2012 .

[30]  Seoyeon Hong,et al.  Computer-mediated persuasion in online reviews: Statistical versus narrative evidence , 2012, Comput. Hum. Behav..

[31]  N. Harrington,et al.  Narrative and Framing: A Test of an Integrated Message Strategy in the Exercise Context , 2011, Journal of health communication.

[32]  Kassandra I. Alcaraz,et al.  Comparing narrative and informational videos to increase mammography in low-income African American women. , 2010, Patient education and counseling.

[33]  B. Nyhan,et al.  When Corrections Fail: The Persistence of Political Misperceptions , 2010 .

[34]  Sahara Byrne,et al.  The Boomerang Effect A Synthesis of Findings and a Preliminary Theoretical Framework , 2009 .

[35]  Robin L. Nabi,et al.  All Joking Aside: A Serious Investigation into the Persuasive Effect of Funny Social Issue Messages , 2007 .

[36]  Raymond W. Preiss,et al.  Meta-Analytic Examination of the Base-Rate Fallacy , 2006 .

[37]  J. Dillard,et al.  On the Nature of Reactance and its Role in Persuasive Health Communication , 2005 .

[38]  K. Greene,et al.  Messages Influencing College Women's Tanning Bed Use: Statistical versus Narrative Evidence Format and a Self-Assessment to Increase Perceived Susceptibility , 2003, Journal of health communication.

[39]  Raymond W. Preiss,et al.  Comparing the persuasiveness of narrative and statistical evidence using meta‐analysis , 1997 .

[40]  Judee K. Burgoon,et al.  The Temporal Effects of Story and Statistical Evidence on Belief Change , 1994 .

[41]  L. Ross,et al.  The hostile media phenomenon: biased perception and perceptions of media bias in coverage of the Beirut massacre. , 1985, Journal of personality and social psychology.

[42]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[43]  Stephen A. Rains.  Human Communication Research Issn 0360-3989 the Nature of Psychological Reactance Revisited: a Meta-analytic Review , 2022 .

[44]  Dean Kazoleas,et al.  A comparison of the persuasive effectiveness of qualitative versus quantitative evidence: A test of explanatory hypotheses , 1993 .

[45]  John T. Cacioppo,et al.  The Elaboration Likelihood Model of Persuasion , 1986, Advances in Experimental Social Psychology.