Visual Framing of Science Conspiracy Videos: Integrating Machine Learning with Communication Theories to Study the Use of Color and Brightness

Recent years have witnessed an explosion of science conspiracy videos on the Internet, challenging science epistemology and public understanding of science. Scholars have started to examine the persuasion techniques used in conspiracy messages such as uncertainty and fear yet, little is understood about the visual narratives, especially how visual narratives differ in videos that debunk conspiracies versus those that propagate conspiracies. This paper addresses this gap in understanding visual framing in conspiracy videos through analyzing millions of frames from conspiracy and counter-conspiracy YouTube videos using computational methods. We found that conspiracy videos tended to use lower color variance and brightness, especially in thumbnails and earlier parts of the videos. This paper also demonstrates how researchers can integrate textual and visual features in machine learning models to study conspiracies on social media and discusses the implications of computational modeling for scholars interested in studying visual manipulation in the digital era. The analysis of visual and textual features presented in this paper could be useful for future studies focused on designing systems to identify conspiracy content on the Internet.

[1]  Savvas Zannettou,et al.  The Good, the Bad and the Bait: Detecting and Characterizing Clickbait on YouTube , 2018, 2018 IEEE Security and Privacy Workshops (SPW).

[2]  Donald P. Greenberg,et al.  Color spaces for computer graphics , 1978, SIGGRAPH.

[3]  Stefan Stieglitz,et al.  Caution: Rumors ahead—A case study on the debunking of false information on Twitter , 2020, Big Data Soc..

[4]  N. Raihani,et al.  An evolutionary perspective on paranoia , 2018, Nature Human Behaviour.

[5]  Alfred Moore Conspiracies, Conspiracy Theories and Democracy , 2018 .

[6]  P. Howard,et al.  Social Media Misinformation on German Intelligence Reports: 'Coronavirus Misinformation Weekly Briefing 18-05-2020' , 2020 .

[7]  R. Norel,et al.  Identifying signals associated with psychiatric illness utilizing language and images posted to Facebook , 2020, npj Schizophrenia.

[8]  Isabel Pinedo Postmodern Elements of the Contemporary Horror Film , 2020, The Horror Film.

[9]  Douglas M. Hawkins,et al.  Assessing Model Fit by Cross-Validation , 2003, J. Chem. Inf. Comput. Sci..

[10]  David Hilbert,et al.  Color and color perception - a study in anthropocentric realism , 1991, CSLI lecture notes series.

[11]  Shujhat Khan,et al.  Coronavirus: the spread of misinformation , 2020, BMC Medicine.

[12]  Lawrence L. Garber,et al.  Color as a Tool for Visual Persuasion , 2003 .

[13]  C. Stemmet Trust no truth : an analysis of the visual translation styles in the conspiracy film , 2012 .

[14]  Joanne M. Miller Psychological, Political, and Situational Factors Combine to Boost COVID-19 Conspiracy Theory Beliefs , 2020, Canadian Journal of Political Science.

[15]  Panic and perjury: a psychosocial exploration of agency. , 2005, The British journal of social psychology.

[16]  A. Cardello,et al.  EFFECTS OF COLORANTS AND FLAVORANTS ON IDENTIFICATION, PERCEIVED FLAVOR INTENSITY, AND HEDONIC QUALITY OF FRUIT‐FLAVORED BEVERAGES AND CAKE , 1980 .

[17]  Mario Cifrek,et al.  A brief introduction to OpenCV , 2012, 2012 Proceedings of the 35th International Convention MIPRO.

[18]  Yaser Sheikh,et al.  On the use of computable features for film classification , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Gernot Wagner,et al.  Solar geoengineering and the chemtrails conspiracy on social media , 2017, Palgrave Communications.

[20]  Fahad Shahbaz Khan,et al.  Transformers in Vision: A Survey , 2021, ACM Comput. Surv..

[21]  Christopher E. Clarke,et al.  The Power of a Picture: Overcoming Scientific Misinformation by Communicating Weight‐of‐Evidence Information with Visual Exemplars , 2015 .

[22]  Vivienne Sze,et al.  Efficient Processing of Deep Neural Networks: A Tutorial and Survey , 2017, Proceedings of the IEEE.

[23]  Joachim Allgaier Science and Environmental Communication on YouTube: Strategically Distorted Communications in Online Videos on Climate Change and Climate Engineering , 2019, Front. Commun..

[24]  Vivek Kumar Singh,et al.  Detecting fake news stories via multimodal analysis , 2020, J. Assoc. Inf. Sci. Technol..

[25]  R. Bromme,et al.  Sealing the gateways for post-truthism: Reestablishing the epistemic authority of science , 2020 .

[26]  Lulu Rodriguez,et al.  The levels of visual framing , 2011 .

[27]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[28]  Yingli Tian,et al.  Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[30]  Yilang Peng What Makes Politicians’ Instagram Posts Popular? Analyzing Social Media Strategies of Candidates and Office Holders with Computer Vision , 2020, The International Journal of Press/Politics.

[31]  Emily K. Vraga,et al.  Testing the Effectiveness of Correction Placement and Type on Instagram , 2020 .

[32]  Ricki Goldman,et al.  Conducting Video Research in the Learning Sciences: Guidance on Selection, Analysis, Technology, and Ethics , 2010 .

[33]  Wasim Ahmed,et al.  COVID-19 and the 5G Conspiracy Theory: Social Network Analysis of Twitter Data , 2020, Journal of Medical Internet Research.

[34]  Sabine Süsstrunk,et al.  Measuring colorfulness in natural images , 2003, IS&T/SPIE Electronic Imaging.

[35]  Corey J. Nolet,et al.  Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence , 2020, Inf..

[36]  Ryan Neville-Shepard Paranoid Style and Subtextual Form in Modern Conspiracy Rhetoric , 2018 .

[37]  Brian P. Meier,et al.  Color in Context: Psychological Context Moderates the Influence of Red on Approach- and Avoidance-Motivated Behavior , 2012, PloS one.

[38]  J. Svensson,et al.  Picturing the Party: Instagram and Party Campaigning in the 2014 Swedish Elections , 2016 .

[39]  Timothy R. Tangherlini,et al.  Conspiracy in the time of corona: automatic detection of emerging COVID-19 conspiracy theories in social media and the news , 2020, Journal of Computational Social Science.

[40]  Andy J. King,et al.  Advancing Visual Health Communication Research to Improve Infodemic Response , 2020, Health communication.

[41]  Emily K. Vraga,et al.  Testing Logic-based and Humor-based Corrections for Science, Health, and Political Misinformation on Social Media , 2019, Journal of Broadcasting & Electronic Media.

[42]  C. Brantner,et al.  Effects of Visual Framing on Emotional Responses and Evaluations of News Stories about the Gaza Conflict 2009 , 2011 .

[43]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[44]  J. Caivano,et al.  How Colour Rhetoric is Used to Persuade: Chromatic Argumentation in Visual Statements , 2010 .

[45]  David Lewis,et al.  Color and Color Perception/A Study in Anthropocentric Realism , 1988 .

[46]  Michael J. Wood,et al.  Propagating and Debunking Conspiracy Theories on Twitter During the 2015–2016 Zika Virus Outbreak , 2018, Cyberpsychology Behav. Soc. Netw..

[47]  Dietram A. Scheufele,et al.  Science audiences, misinformation, and fake news , 2019, Proceedings of the National Academy of Sciences.

[48]  Daniel Kreiss,et al.  The Marketplace of Attention : How Audiences Take Shape in a Digital Age , 2016 .

[49]  Jungseock Joo,et al.  Image as Data: Automated Visual Content Analysis for Political Science , 2018, ArXiv.

[50]  Karen M. Douglas,et al.  Conspiracy theories as part of history: The role of societal crisis situations , 2017, Memory studies.

[51]  Theo van Leeuwen,et al.  Reading Images: The Grammar of Visual Design , 1996 .

[52]  Wen-Hsing Hsu,et al.  Film classification based on low-level visual effect features , 2008, J. Electronic Imaging.

[53]  Steve Osborne ‘The total package’ , 1999 .

[54]  James E. Katz,et al.  From Belief in Conspiracy Theories to Trust in Others: Which Factors Influence Exposure, Believing and Sharing Fake News , 2019, HCI.

[55]  Nitin Agarwal,et al.  Analyzing Disinformation and Crowd Manipulation Tactics on YouTube , 2018, 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[56]  Fatima Afifah,et al.  Vehicle Speed Estimation using Image Processing , 2019 .

[57]  Diana Rieger,et al.  Counter-messages as Prevention or Promotion of Extremism?! The Potential Role of YouTube , 2018, Journal of Communication.

[58]  David G. Rand,et al.  Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention , 2020, Psychological science.

[59]  Yilang Peng,et al.  Feast for the Eyes: Effects of Food Perceptions and Computer Vision Features on Food Photo Popularity , 2018 .

[60]  Stef Aupers,et al.  ‘Trust no one’: Modernization, paranoia and conspiracy culture , 2012 .

[61]  Yilang Peng,et al.  Same Candidates, Different Faces: Uncovering Media Bias in Visual Portrayals of Presidential Candidates with Computer Vision , 2018, Journal of Communication.

[62]  Karen M. Douglas,et al.  Climate change: Why the conspiracy theories are dangerous , 2015 .

[63]  P. Valdez,et al.  Effects of color on emotions. , 1994, Journal of experimental psychology. General.

[64]  James N. Druckman,et al.  F RAMING T HEORY , 2007 .

[65]  E. Oxman Sensing the Image: Roland Barthes and the Affect of the Visual , 2010 .

[66]  Yubo Kou Conspiracy Talk on Social Media: Collective Sensemaking during a Public Health Crisis , 2017 .

[67]  E. Bucy,et al.  Editors’ Introduction: Visual Politics, Grand Collaborative Programs, and the Opportunity to Think Big , 2020, The International Journal of Press/Politics.

[68]  Steve Fotios,et al.  Measuring Colour , 2013 .

[69]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[70]  Eva M. Hyatt,et al.  The Effects of Food Color on Perceived Flavor , 2000 .

[71]  R. Kelly Garrett,et al.  Undermining the Corrective Effects of Media‐Based Political Fact Checking? The Role of Contextual Cues and Naïve Theory , 2013 .

[72]  Sebastian Raschka,et al.  Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning , 2018, ArXiv.

[73]  Mary A. Gerend,et al.  Message framing and color priming: How subtle threat cues affect persuasion , 2009 .

[74]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[75]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .