Ratings Use in an Online Discussion System: The Slashdot Case

Large-scale online communities need to manage the tension between critical mass and information overload. Slashdot is a news and discussion site that has used comment rating to allow massive participation while providing a mechanism for users to filter content. We find that many users are actively using ratings to restructure how comments are shown to them. However, many users are not using ratings to guide their reading behavior, which may be caused by friction: the perceived cost of changing a setting versus the perceived benefit for having done so. We recommend leveraging the efforts of the users that do use the rating system to reduce the cost of changing for all other users. One strategy is the creation of statistically motivated schemas that represent types of reading behavior. Another strategy is to dynamically set filtering thresholds for comments, based in part on the choices of previous readers, which are far better predictors of readers’ choices than content features such as the number of comments or the ratings of those comments. 12 Work done with Erik Johnston and Paul Resnick, pending submission.

[1]  E. Friedman,et al.  The Social Cost of Cheap Pseudonyms , 2001 .

[2]  James D. Hollan,et al.  Beyond being there , 1992, CHI.

[3]  Paul Resnick,et al.  Reputation systems , 2000, CACM.

[4]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[5]  Sorin Adam Matei,et al.  From Counterculture to Cyberculture: Virtual Community Discourse and the Dilemma of Modernity , 2006, J. Comput. Mediat. Commun..

[6]  T. Christiano.  The Significance of Public Deliberation , 1997 .

[7]  W. Galston.  Does the Internet Strengthen Community , 1999 .

[8]  P. Resnick.  Beyond Bowling Together: SocioTechnical Capital , 2001 .

[9]  Joshua R. Smith,et al.  Debate, Division, and Diversity: Political Discourse Networks in USENET Newsgroups , 2005 .

[10]  Amy Jo Kim,et al.  Community Building on the Web: Secret Strategies for Successful Online Communities , 2000 .

[11]  Nathaniel Poor,et al.  Mechanisms of an Online Public Sphere: The Website Slashdot , 2006, J. Comput. Mediat. Commun..

[12]  Stuart E. Middleton,et al.  Ontological user profiling in recommender systems , 2004, TOIS.

[13]  Joanna McGrenere,et al.  A comparison of static, adaptive, and adaptable menus , 2004, CHI.

[14]  David Crystal,et al.  Language and the Internet , 2001 .

[15]  C. Mcphail.  The myth of the madding crowd , 1991 .

[16]  Sheizaf Rafaeli,et al.  User population and user contributions to virtual publics: a systems model , 1999, GROUP.

[17]  M. Taylor,et al.  Consequences of individual feedback on behavior in organizations. , 1979 .

[18]  William A. Galston.  Does the Internet Strengthen Community , 1999 .

[19]  S. Bikhchandani,et al.  Learning from the behavior of others : conformity, fads, and informational cascades , 1998 .

[20]  Chrysanthos Dellarocas,et al.  The Digitization of Word-of-Mouth: Promise and Challenges of Online Feedback Mechanisms , 2003, Manag. Sci..

[21]  Cliff Lampe,et al.  Follow the (slash) dot: effects of feedback on new members in an online community , 2005, GROUP.

[22]  Loren G. Terveen,et al.  The dynamics of mass interaction , 1998, CSCW '98.

[23]  Howard Rheingold,et al.  Smart Mobs: The Next Social Revolution , 2002 .

[24]  Starr Roxanne Hiltz,et al.  Network Nation: Human Communication Via Computer , 1979 .

[25]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[26]  Pier Luca Lanzi,et al.  Mining interesting knowledge from weblogs: a survey , 2005, Data Knowl. Eng..

[27]  S. Turkle.  Life on the Screen: Identity in the Age of the Internet , 1997 .

[28]  Marc A. Smith,et al.  Visualization components for persistent conversations , 2001, CHI.

[29]  Marc A. Smith,et al.  Measures and Maps of Usenet , 2003, From Usenet to CoWebs.

[30]  Paul Resnick,et al.  Slash(dot) and burn: distributed moderation in a large online conversation space , 2004, CHI.

[31]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[32]  Martin Wattenberg,et al.  Studying cooperation and conflict between authors with history flow visualizations , 2004, CHI.

[33]  Judith S. Olson,et al.  Distance Matters , 2000, Hum. Comput. Interact..

[34]  Loren Terveen,et al.  Beyond Recommender Systems: Helping People Help Each Other , 2001 .

[35]  Martin Wattenberg,et al.  Flash forums and forumReader: navigating a new kind of large-scale online discussion , 2004, CSCW.

[36]  Merrill Morris.  E-Mail Editors: Gatekeepers or Facilitators?. , 1993 .

[37]  Marc A. Smith.  Invisible Crowds in Cyberspace: Mapping the Social Structure of the Usenet , 1999 .

[38]  Anna Smith.  Problems of conflict management in virtual communities , 2002 .

[39]  D. Gillmor.  We the Media , 2004 .

[40]  H. H. Clark.  Arenas of language use , 1993 .

[41]  Derek Powazek,et al.  Design for Community: The Art of Connecting Real People in Virtual Places , 2001 .

[42]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[43]  Les Gasser,et al.  Negotiation and the coordination of information and activity in distributed software problem management , 2005, GROUP.

[44]  Gilad Ravid,et al.  An empirical exploration of mass interaction system dynamics: individual information overload and Usenet discourse , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[45]  P. Resnick,et al.  The Market for Evaluations , 1999 .

[46]  John S. Quarterman,et al.  Notable computer networks , 1991 .

[47]  J. G. Miller,et al.  Information input overload and psychopathology. , 1960, The American journal of psychiatry.

[48]  Mark S. Ackerman,et al.  I-DIAG: from community discussion to knowledge distillation , 2003 .

[49]  Eric Lease Morgan,et al.  Review of The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary by Eric S. Raymond, Sebastopol, Calif.: O'Reilly, 1999 , 2000 .

[50]  Pattie Maes,et al.  Social information filtering: algorithms for automating “word of mouth” , 1995, CHI '95.

[51]  Lada A. Adamic,et al.  The political blogosphere and the 2004 U.S. election: divided they blog , 2005, LinkKDD '05.

[52]  Gavriel Salomon,et al.  T RANSFER OF LEARNING , 1992 .

[53]  Paul Dourish,et al.  Social and temporal structures in everyday collaboration , 2004, CHI.

[54]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[55]  T. Landauer.  Let's get real: a position paper on the role of cognitive psychology in the design of humanly useful and usable systems , 1991 .

[56]  Pavel Curtis,et al.  Mudding: Social phenomena in text-based virtual realities. , 1997 .

[57]  B. Butler.  When is a Group not a Group: An Empirical Examination of Metaphors for Online Social Structure , 2003 .

[58]  C. Bray.  Transfer of learning. , 1928 .

[59]  Quentin Jones,et al.  Virtual-Communities, Virtual Settlements & Cyber-Archaeology: A Theoretical Outline , 2006, J. Comput. Mediat. Commun..

[60]  Wendy E. Mackay,et al.  Triggers and barriers to customizing software , 1991, CHI.

[61]  Kipling D. Williams,et al.  Interpersonal Relations and Group Processes Social Loafing: a Meta-analytic Review and Theoretical Integration , 2022 .

[62]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[63]  J. Gillon,et al.  Group dynamics , 1996 .

[64]  Bradley N. Miller,et al.  Applying Collaborative Filtering to Usenet News , 1997 .

[65]  John Riedl,et al.  Is seeing believing?: how recommender system interfaces affect users' opinions , 2003, CHI '03.

[66]  Pavel Curtis,et al.  MUDs grow up: social virtual reality in the real world , 1994, Proceedings of COMPCON '94.

[67]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[68]  Ken Thompson,et al.  The UNIX time-sharing system , 1974, CACM.

[69]  Loren G. Terveen,et al.  Using frequency-of-mention in public conversations for social filtering , 1996, CSCW '96.

[70]  Alexander Halavais,et al.  The slashdot effect : analysis of a large-scale public conversation on the World Wide Web , 2001 .

[71]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[72]  Fernanda B. Viégas,et al.  Newsgroup Crowds and AuthorLines: visualizing the activity of individuals in conversational cyberspaces , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[73]  Thomas P. Moran,et al.  User-tailorable systems: pressing the issues with buttons , 1990, CHI '90.

[74]  Katie Hafner.  Where Wizards Stay Up Late , 1996 .

[75]  Franco Scarselli,et al.  Inside PageRank , 2005, TOIT.

[76]  J. Horrigan.  Online communities: Networks that nurture long-distance relationships and local ties , 2001 .

[77]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[78]  Gerhard Weiss.  Groupware and Computer Supported Cooperative Work , 2000 .

[79]  A. Bandura.  Aggression: a social learning analysis , 1971 .

[80]  Gerhard Fischer,et al.  Beyond intelligent interfaces: Exploring, analyzing, and creating success models of cooperative problem solving , 1992, Applied Intelligence.

[81]  M. McLure Wasko,et al.  "It is what one does": why people participate and help others in electronic communities of practice , 2000, J. Strateg. Inf. Syst..

[82]  E. Goffman.  Behavior in public places : notes on the social organization of gatherings , 1964 .

[83]  Jennifer Preece,et al.  The top five reasons for lurking: improving community experiences for everyone , 2004, Comput. Hum. Behav..

[84]  David M. Pennock,et al.  Categories and Subject Descriptors , 2001 .

[85]  Organizations , 1992, Restoration & Management Notes.

[86]  Brian S. Butler,et al.  Membership Size, Communication Activity, and Sustainability: A Resource-Based Model of Online Social Structures , 2001, Inf. Syst. Res..

[87]  C. Dennis Allen,et al.  User customization of a word processor , 1996, CHI.

[88]  Mark S. Ackerman,et al.  Computing, Social Activity, and Entertainment: A Field Study of a Game MUD , 2004, Computer Supported Cooperative Work (CSCW).

[89]  John Riedl,et al.  Meta-recommendation systems: user-controlled integration of diverse recommendations , 2002, CIKM '02.

[90]  Kamal Ali,et al.  TiVo: making show recommendations using a distributed collaborative filtering architecture , 2004, KDD.

[91]  D. Hindman.  The Virtual Community: Homesteading on the Electronic Frontier , 1996 .

[92]  B. Wellman.  An electronic group is virtually a social network. , 1997 .

[93]  G. Olson,et al.  From Laboratories to Collaboratories: A New Organizational Form for Scientific Collaboration , 1997 .

[94]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[95]  Warren Sack,et al.  Conversation Map: An Interface for Very Large-Scale Conversations , 2000, J. Manag. Inf. Syst..

[96]  S. Bikhchandani,et al.  You have printed the following article : A Theory of Fads , Fashion , Custom , and Cultural Change as Informational Cascades , 2007 .

[97]  M. Couper.  A REVIEW OF ISSUES AND APPROACHES , 2000 .

[98]  Starr Roxanne Hiltz,et al.  Structuring computer-mediated communication systems to avoid information overload , 1985, CACM.

[99]  S. Horn.  Cyberville: Clicks, Culture, and the Creation of an Online Town , 1998 .

[100]  Amy Bruckman,et al.  Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia , 2005, GROUP.

[101]  Bryan Pfaffenberger,et al.  "A Standing Wave in the Web of Our Communications": Usenet and the Socio-Technical Construction of Cyberspace Values , 2003, From Usenet to CoWebs.

[102]  Maureen S. Battistella,et al.  Connections: New Ways of Working in the Networked Organization , 1991 .

[103]  Gerhard Fischer,et al.  End-user modifiability in design environments , 1990, CHI '90.

[104]  Mitchel Resnick,et al.  Moose crossing: construction, community, and learning in a networked virtual world for kids , 1997 .

[105]  Scott LeeTiernan,et al.  Observed behavior and perceived value of authors in usenet newsgroups: bridging the gap , 2002, CHI.

[106]  A. Banerjee,et al.  A Simple Model of Herd Behavior , 1992 .

[107]  Mark S. Ackerman,et al.  Answer Garden 2: merging organizational memory with collaborative help , 1996, CSCW '96.

[108]  P. Kollock,et al.  Managing the virtual commons : Cooperation and conflict in computer communities , 1996 .

[109]  H. Grice.  Utterer's meaning and intentions , 1969 .

[110]  J. Abbate,et al.  Inventing the Internet , 1999 .