Normative Change and Culture of Hate: An Experiment in Online Environments

We present an online experiment in which we investigate the impact of perceived social acceptability on online hate speech, and measure the causal effect of specific interventions. We compare two types of interventions: counter-speaking (informal verbal sanctions) and censoring (deleting hateful content). The interventions are based on the belief that individuals infer acceptability from the context, using previous actions as a source of normative information. The interventions are based on the two conceptualizations found in the literature: 1) what do others normally do, i.e., descriptive norms; and 2) what happened to those who violated the norm, i.e., injunctive norms. Participants were significantly less likely to engage in hate speech when prior hate content had been moderately censored. Our results suggest that normative behavior in online conversations might, in fact, be motivated by descriptive norms rather than injunctive norms. With this work we present some of the first experimental evidence investigating the social determinants of hate speech in online communities. The results could advance the understanding of the micro-mechanisms that regulate hate speech. Also, such findings can guide future interventions in online communities that help prevent the spread of hate.

[1]  Michael S. Bernstein,et al.  Anyone Can Become a Troll , 2017 .

[2]  D. Green,et al.  Prejudice reduction: what works? A review and assessment of research and practice. , 2009, Annual review of psychology.

[3]  Gerard Delanty.  The Foundations of Social Theory , 2009 .

[4]  Melanie Taylor,et al.  Online Harassment and Victimization of College Students , 2010 .

[5]  P. Resnick,et al.  Building Successful Online Communities: Evidence-Based Social Design , 2012 .

[6]  Edward Cartwright,et al.  On the Emergence of Social Norms , 2007 .

[7]  Scott Blinder,et al.  The Anti‐Racism Norm in Western European Immigration Politics: Why we Need to Consider it and How to Measure it , 2010 .

[8]  Justin W. Patchin,et al.  Offline Consequences of Online Victimization , 2007 .

[9]  Jenessa R. Shapiro,et al.  When do the stigmatized stigmatize? The ironic effects of being accountable to (perceived) majority group prejudice-expression norms. , 2008, Journal of personality and social psychology.

[10]  Noah J. Goldstein,et al.  Social influence: compliance and conformity. , 2004, Annual review of psychology.

[11]  S. Fiske,et al.  The Handbook of Social Psychology , 1935 .

[12]  J. List,et al.  Voting to Tell Others , 2014 .

[13]  Kimberly A. Neuendorf,et al.  Reliability for Content Analysis , 2010 .

[14]  Dynamic Bargaining and Normative Conflict , 2018, Journal of Behavioral and Experimental Economics.

[15]  Helen L. Norton,et al.  Intermediaries and Hate Speech: Fostering Digital Citizenship for Our Information Age , 2011 .

[16]  D. A. Kenny,et al.  Experiments with More Than One Random Factor: Designs, Analytic Models, and Statistical Power , 2017, Annual review of psychology.

[17]  M. Hebl,et al.  The role of social norm clarity in the influenced expression of prejudice over time , 2007 .

[18]  C. Stangor,et al.  Conformity and Prejudice , 2008 .

[19]  C. Bicchieri,et al.  When Equality Trumps Reciprocity , 2010 .

[20]  H. Young,et al.  Social Norms , 2021, Encyclopedia of Evolutionary Psychological Science.

[21]  Cristina Bicchieri,et al.  Do the Right Thing: But Only If Others Do So , 2007 .

[22]  J Tudor-Hart,et al.  On the nature of prejudice. , 1961, The Eugenics review.

[23]  R. Cialdini,et al.  Social influence: Social norms, conformity and compliance. , 1998 .

[24]  Bruno S. Frey,et al.  Digital Social Norm Enforcement: Online Firestorms in Social Media , 2016, PloS one.

[25]  Robert C. Ford.  Is racial prejudice declining in Britain? , 2008, The British journal of sociology.

[26]  Charles Stangor,et al.  Changing Racial Beliefs by Providing Consensus Information , 2001 .

[27]  Georgy Egorov,et al.  From Extreme to Mainstream: How Social Norms Unravel , 2017 .

[28]  Mantilla Karla.  Gendertrolling: Misogyny Adapts to New Media , 2013 .

[29]  M. Sherif,et al.  Groups in harmony and tension : an integration of studies on intergroup relations , 1954 .

[30]  C. Crandall,et al.  Social norms and the expression and suppression of prejudice: the struggle for internalization. , 2002, Journal of personality and social psychology.

[31]  A. Binns.  DON'T FEED THE TROLLS! , 2012 .

[32]  T. Pettigrew.  Personality and sociocultural factors in intergroup attitudes: a cross-national comparison , 1958 .

[33]  K. Opp.  Social Institutions: Their Emergence, Maintenance, and Effects , 2018 .

[34]  J. Brigham,et al.  Condemning and condoning racism : a social context approach to interracial settings , 1994 .

[35]  Laurie A. Rudman,et al.  On the nature of prejudice : fifty years after Allport , 2005 .

[36]  C. Bicchieri.  The grammar of society: the nature and dynamics of social norms , 2005 .

[37]  Scott Blinder,et al.  The Better Angels of Our Nature: How the Antiprejudice Norm Affects Policy and Party Preferences in Great Britain and Germany , 2013 .

[38]  Björn Ross,et al.  Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis , 2016, ArXiv.

[39]  K. Krippendorff.  Reliability in Content Analysis: Some Common Misconceptions and Recommendations , 2004 .

[40]  J. Dovidio,et al.  Prejudice, discrimination, and racism: Historical trends and contemporary approaches. , 1986 .

[41]  Wolfgang J. Luhan,et al.  Experimental political science : principles and practices , 2012 .

[42]  Michael S. Bernstein,et al.  Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions , 2017, CSCW.

[43]  Elizabeth Levy Paluck,et al.  Norm Perception as a Vehicle for Social Change , 2016 .

[44]  Bradford W Reyns,et al.  Fear of Crime Online? Examining the Effect of Risk, Previous Victimization, and Exposure on Fear of Online Interpersonal Victimization , 2013 .

[45]  Michael Burgoon,et al.  Revisiting the Theory of Psychological Reactance: Communicating Threats to Attitudinal Freedom , 2002 .

[46]  Erin L. Krupka,et al.  The Focusing and Informational Effects of Norms on Pro-Social Behavior , 2007, SSRN Electronic Journal.

[47]  Kevin Munger.  Tweetment Effects on the Tweeted: Experimentally Reducing Racist Harassment , 2017 .

[48]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[49]  Tobias Wolbring,et al.  Disorder, social capital, and norm violation: Three field experiments on the broken windows thesis , 2015 .

[50]  Erin L. Krupka,et al.  Identifying Social Norms Using Coordination Games: Why Does Dictator Game Sharing Vary? Identifying Social Norms Using Coordination Games: Why Does Dictator Game Sharing Vary? Identifying Social Norms Using Coordination Games: Why Does Dictator Game Sharing Vary? , 2022 .

[51]  J. Elster.  Social Norms and Economic Theory , 1989, Handbook of Monetary Policy.

[52]  Cristina Bicchieri,et al.  NORM MANIPULATION, NORM EVASION: EXPERIMENTAL EVIDENCE , 2011, Economics and Philosophy.

[53]  Daniel L. Chen,et al.  oTree - An Open-Source Platform for Laboratory, Online, and Field Experiments , 2016 .

[54]  C. Bicchieri,et al.  SHRIEKING SIRENS: SCHEMATA, SCRIPTS, AND SOCIAL NORMS. HOW CHANGE OCCURS , 2018, Social Philosophy and Policy.

[55]  Carl A. Kallgren,et al.  A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. , 1990 .

[56]  Edna Ullmann-Margalit,et al.  The Emergence Of Norms , 1977 .

[57]  Linda Steg,et al.  The Spreading of Disorder , 2008, Science.

[58]  T. Pettigrew.  Normative Theory in Intergroup Relations: Explaining Bo th Harmony an d Conflict , 1991 .

[59]  Douglas D. Heckathorn,et al.  Collective Sanctions and the Creation of Prisoner's Dilemma Norms , 1988, American Journal of Sociology.

[60]  Tom A. B. Snijders,et al.  Multilevel Analysis , 2011, International Encyclopedia of Statistical Science.

[61]  J. Duckitt.  Psychology and prejudice: A historical analysis and integrative framework. , 1992 .

[62]  E. Fehr,et al.  Cooperation and Punishment in Public Goods Experiments , 1999, SSRN Electronic Journal.