Digital Juries: A Civics-Oriented Approach to Platform Governance

As concerns have grown regarding harmful content spread on social media, platform mechanisms for content moderation have become increasingly significant. However, many existing platform governance structures lack formal processes for democratic participation by users of the platform. Drawing inspiration from constitutional jury trials in many legal systems, this paper proposes digital juries as a civics-oriented approach for adjudicating content moderation cases. Building on existing theoretical models of jury decision-making, we outline a 5-stage model characterizing the space of design considerations in a digital jury process. We implement two examples of jury designs involving blind-voting and deliberation. From users who participate in our jury implementations, we gather informed judgments of the democratic legitimacy of a jury process for content moderation. We find that digital juries are perceived as more procedurally just than existing common platform moderation practices, but also find disagreement over whether jury decisions should be enforced or used as recommendations.

[1]  T. Quandt Behind the screen: content moderation in the shadows of social media , 2020, Digital Journalism.

[2]  Casey Fiesler,et al.  Reddit Rules! Characterizing an Ecosystem of Governance , 2018, ICWSM.

[3]  Nigel Harvey,et al.  Blackwell Handbook of Judgment and Decision Making , 2004 .

[4]  Amy Bruckman,et al.  Approaches to managing deviant behavior in virtual communities , 1994, CHI Conference Companion.

[5]  Brian S. Butler,et al.  Don't look now, but we've created a bureaucracy: the nature and roles of policies and rules in wikipedia , 2008, CHI.

[6]  Mary J. Newhart,et al.  Democratic Deliberation in the Wild: The McGill Online Design Studio and the RegulationRoom Project , 2014 .

[7]  P. Resnick Beyond Bowling Together: SocioTechnical Capital , 2001 .

[8]  David G. Post Anarchy, State, and the Internet: An Essay on Law-Making in Cyberspace (article 3) , 2006 .

[9]  P. Resnick,et al.  Building Successful Online Communities: Evidence-Based Social Design , 2012 .

[10]  Seungyeop Han,et al.  Exploring Cyberbullying and Other Toxic Behavior in Team Competition Online Games , 2015, CHI.

[11]  Nicole B. Ellison,et al.  Harassment in Social Virtual Reality , 2019, Proc. ACM Hum. Comput. Interact..

[12]  J. Nathan Matias,et al.  The Civic Labor of Volunteer Moderators Online , 2019, Social Media + Society.

[13]  C. Nemeth,et al.  Interactions between jurors as a function of majority vs. unanimity decision rules. , 1977 .

[14]  Alan Borning,et al.  Supporting reflective public thought with considerit , 2012, CSCW.

[15]  Kate Klonick,et al.  The New Governors: The People, Rules, and Processes Governing Online Speech , 2017 .

[16]  E. Brink,et al.  Constructing grounded theory : A practical guide through qualitative analysis , 2006 .

[17]  Joshua M. Epstein,et al.  Agent_Zero: Toward Neurocognitive Foundations for Generative Social Science , 2014 .

[18]  J. Eichel Victims Of Groupthink A Psychological Study Of Foreign Policy Decisions And Fiascoes , 2016 .

[19]  Kimberley R. Allison,et al.  'I'm going to hell for laughing at this' , 2019, Proc. ACM Hum. Comput. Interact..

[20]  ชวิตรา ตันติมาลา Constructing Grounded Theory: A Practical Guide through Qualitative Analysis , 2017 .

[21]  Bonnie A. Nardi,et al.  Managing Disruptive Behavior through Non-Hierarchical Governance , 2017, Proc. ACM Hum. Comput. Interact..

[22]  David R. Karger,et al.  Deliberation and Resolution on Wikipedia , 2018, Proceedings of the ACM on Human-Computer Interaction.

[23]  Eric Gilbert,et al.  The Internet's Hidden Rules , 2018, Proceedings of the ACM on Human-Computer Interaction.

[24]  Amy Bruckman,et al.  Decentralization in Wikipedia Governance , 2009, J. Manag. Inf. Syst..

[25]  Cameron Marlow,et al.  A 61-million-person experiment in social influence and political mobilization , 2012, Nature.

[26]  Seth Frey,et al.  "This Place Does What It Was Built For" , 2019, Proc. ACM Hum. Comput. Interact..

[27]  Jonathan T. Morgan,et al.  The Rise and Decline of an Open Collaboration System , 2013 .

[28]  Mark Zuckerberg,et al.  MZ shares a note - A Blueprint for Content Governance and Enforcement , 2018 .

[29]  T. Tyler The psychology of procedural justice: A test of the group-value model. , 1989 .

[30]  The Vanishing Juror: Why Are There Not Enough Available Jurors?* , 2016 .

[31]  Lydia B. Chilton,et al.  MicroTalk: Using Argumentation to Improve Crowdsourcing Accuracy , 2016, HCOMP.

[32]  Bonnie A. Nardi,et al.  Governance in League of Legends: A hybrid system , 2014, FDG.

[33]  Miriam Solomon,et al.  Groupthink versus The Wisdom of Crowds: The Social Epistemology of Deliberation and Dissent , 2006 .

[34]  John Gastil,et al.  Deliberative Democracy and the American Civil Jury , 2014 .

[35]  N. Pennington,et al.  Inside the Jury. , 1985 .

[36]  T. Tyler,et al.  The Social Psychology of Procedural Justice , 1988 .

[37]  Jeffery J. Mondak,et al.  Jury Service as Civic Engagement , 2012 .

[38]  Lynne Tirrell,et al.  Toxic Speech: Inoculations and Antidotes , 2018, The Southern Journal of Philosophy.

[39]  Ysabel Gerrard,et al.  Behind the Screen: Content Moderation in the Shadows of Social Media , 2019, New Media Soc..

[40]  Kate Crawford,et al.  What is a flag for? Social media reporting tools and the vocabulary of complaint , 2016, New Media Soc..

[41]  Lawrence Lessig,et al.  Code - version 2.0 , 2006 .

[42]  Martin J. Riedl,et al.  Book review: Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media , 2019, New Media Soc..

[43]  Radha Poovendran,et al.  Deceiving Google's Perspective API Built for Detecting Toxic Comments , 2017, ArXiv.

[44]  Joseph A. Hamm,et al.  Public Participation, Procedural Fairness, and Evaluations of Local Governance: The Moderating Role of Uncertainty , 2012 .

[45]  Darren Gergle,et al.  Effects of four computer-mediated communications channels on trust development , 2002, CHI.

[46]  Peter Muhlberger,et al.  The Virtual Agora Project: A Research Design for Studying Democratic Deliberation , 2005 .

[47]  Bonnie A. Nardi,et al.  Regulating anti-social behavior on the Internet: The example of League of Legends , 2013 .

[48]  Seth Frey,et al.  Emergence of integrated institutions in a large population of self-governing communities , 2018, PloS one.

[49]  Lydia B. Chilton,et al.  Cicero: Multi-Turn, Contextual Argumentation for Accurate Crowdsourcing , 2018, CHI.

[50]  Tom R. Tyler,et al.  What is procedural justice? Criteria used by citizens to assess the fairness of legal procedures. , 1988 .

[51]  Edith Greene,et al.  Juror Decision‐Making , 2008 .

[52]  Edith Law,et al.  Resolvable vs. Irresolvable Disagreement , 2018, Proc. ACM Hum. Comput. Interact..

[53]  Richard L. Daft,et al.  Organizational information requirements, media richness and structural design , 1986 .

[54]  Robyn Caplan Content or Context Moderation , 2018 .

[55]  Stephan Hartmann,et al.  Voting, deliberation and truth , 2016, Synthese.

[56]  John Gastil The Jury and Democracy: How Jury Deliberation Promotes Civic Engagement and Political Participation , 2010 .

[57]  Tarleton Gillespie,et al.  Platforms Intervene , 2015 .

[58]  Jacob Eisenstein,et al.  You Can't Stay Here , 2017 .

[59]  G. Brady Governing the Commons: The Evolution of Institutions for Collective Action , 1993 .

[60]  C. Sunstein The Law of Group Polarization , 1999, How Change Happens.

[61]  Nicolas P. Suzor Digital Constitutionalism: Using the Rule of Law to Evaluate the Legitimacy of Governance by Platforms , 2016 .

[62]  S. Kaplan,et al.  The occupational hazards of jury duty. , 1992, The Bulletin of the American Academy of Psychiatry and the Law.

[63]  N. Dalkey,et al.  An Experimental Application of the Delphi Method to the Use of Experts , 1963 .

[64]  Christian List,et al.  Disaggregating Deliberation’s Effects: An Experiment within a Deliberative Poll , 2010, British Journal of Political Science.

[65]  Lee Sproull,et al.  Prosocial Behavior on the Net , 2011, Daedalus.

[66]  Panagiotis G. Ipeirotis,et al.  Demographics and Dynamics of Mechanical Turk Workers , 2018, WSDM.

[67]  Loren G. Terveen,et al.  The Success and Failure of Quality Improvement Projects in Peer Production Communities , 2015, CSCW.

[68]  Michael Veale,et al.  Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation , 2017, SocInfo.

[69]  M. D. Carpini,et al.  Public deliberation, discursive participation, and citizen engagement: A review of the empirical literature , 2004 .

[70]  Gianluca Stringhini,et al.  Screenshot Classifier annotated images pHashes of non-screenshot annotated images Know Your Meme Generic Annotation Sites Meme Annotation Sites Generic Web Communities , 2018 .

[71]  Joshua A. Tucker,et al.  Emotion shapes the diffusion of moralized content in social networks , 2017, Proceedings of the National Academy of Sciences.

[72]  Yubo Kou,et al.  When Code Governs Community , 2017, HICSS.

[73]  Jack M. Balkin,et al.  Virtual Liberty: Freedom to Design and Freedom to Play in Virtual Worlds , 2004 .

[74]  S. Kiesler,et al.  Group decision making and communication technology , 1992 .

[75]  David R. Karger,et al.  Wikum: Bridging Discussion Forums and Wikis Using Recursive Summarization , 2017, CSCW.

[76]  I. Janis Victims of Groupthink: A psychological study of foreign-policy decisions and fiascoes. By Irving L. Janis. (Boston: Houghton Mifflin, 1972. viii + 276 pp. Map, illustrations, chart, notes, sources, bibliography, and index. Cloth, $7.95; paper $4.50.) , 1973 .

[77]  Robert E. Kraut,et al.  Mopping up: modeling wikipedia promotion decisions , 2008, CSCW.

[78]  Paul Resnick,et al.  Slash(dot) and burn: distributed moderation in a large online conversation space , 2004, CHI.

[79]  Cliff Lampe,et al.  Classification and Its Consequences for Online Harassment , 2017, Proc. ACM Hum. Comput. Interact..

[80]  James Grimmelmann,et al.  The Virtues of Moderation , 2015 .

[81]  Sarah T. Roberts Commercial Content Moderation: Digital Laborers' Dirty Work , 2016 .

[82]  Peter H. Ditto,et al.  Moral Foundations Theory: The Pragmatic Validity of Moral Pluralism , 2012 .