Research Note - Content and Collaboration: An Affiliation Network Approach to Information Quality in Online Peer Production Communities

The 15-year history of collaboration on Wikipedia offers insight into how peer production communities create knowledge. In this research, we combine disparate content and collaboration approaches through a social network analysis approach known as an affiliation network. It captures both how knowledge is transferred in a peer production network and also the underlying skills possessed by its contributors in a single methodological approach. We test this approach on the Wikipedia articles dedicated to medical information developed in a subcommunity known as a WikiProject. Overall, we find that the position of an article in the affiliation network is associated with the quality of the article. We further investigate information quality through additional qualitative and quantitative approaches including expert coders using medical students, crowdsourcing using Amazon Mechanical Turk, and visualization using network graphs. A review by fourth-year medical students indicates that the Wikipedia quality rating is a reliable measure of information quality. Amazon Mechanical Turk ratings, however, are a less reliable measure of information quality, reflecting observable content characteristics such as article length and the number of references.

[1]  Ulrik Brandes,et al.  Network analysis of collaboration structure in Wikipedia , 2009, WWW '09.

[2]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[3]  Aniket Kittur,et al.  Beyond Wikipedia: coordination and conflict in online production groups , 2010, CSCW '10.

[4]  S. Kiesler,et al.  The kindness of strangers: on the usefulness of electronic weak ties for technical advice , 1996 .

[5]  Daniel J. Brass,et al.  Exploring the Social Ledger: Negative Relationships and Negative Asymmetry in Social Networks in Organizations , 2006 .

[6]  Aniket Kittur,et al.  Coordination in collective intelligence: the role of team structure and task interdependence , 2009, CHI.

[7]  X. Zhang,et al.  Group Size and Incentives to Contribute: A Natural Experiment at Chinese Wikipedia , 2010 .

[8]  W. Cook,et al.  Ordinal Ranking with Intensity of Preference , 1985 .

[9]  Guido Caldarelli,et al.  Preferential attachment in the growth of social networks: the case of Wikipedia , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  P. Gloor,et al.  Analyzing the Creative Editing Behavior of Wikipedia Editors: Through Dynamic Social Network Analysis , 2010 .

[11]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[12]  George Kuk,et al.  Strategic Interaction and Knowledge Sharing in the KDE Developer Mailing List , 2006, Manag. Sci..

[13]  Bernardo A. Huberman,et al.  Cooperation and quality in wikipedia , 2007, WikiSym '07.

[14]  M. Speekenbrink,et al.  depmixS4: An R Package for Hidden Markov Models , 2010 .

[15]  Phillip Bonacich,et al.  Some unique properties of eigenvector centrality , 2007, Soc. Networks.

[16]  Brian S. Butler,et al.  The Impact of Membership Overlap on Growth: An Ecological Competition View of Online Groups , 2013, Organ. Sci..

[17]  Brian S. Butler,et al.  Membership Size, Communication Activity, and Sustainability: A Resource-Based Model of Online Social Structures , 2001, Inf. Syst. Res..

[18]  Ulrike Cress,et al.  Who integrates the networks of knowledge in Wikipedia? , 2010, Int. Sym. Wikis.

[19]  Matthias Trier,et al.  Research Note - Towards Dynamic Visualization for Understanding Evolution of Digital Communication Networks , 2008, Inf. Syst. Res..

[20]  J. March Exploration and exploitation in organizational learning , 1991, STUDI ORGANIZZATIVI.

[21]  Ramayya Krishnan,et al.  An Empirical Analysis of Network Externalities in Peer-to-Peer Music Sharing Networks , 2003, ICIS.

[22]  Aniket Kittur,et al.  Harnessing the wisdom of crowds in wikipedia: quality through coordination , 2008, CSCW.

[23]  Paolo Massa,et al.  Social networks of Wikipedia , 2011, HT '11.

[24]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[25]  Ann Majchrzak,et al.  Knowledge Collaboration in Online Communities , 2011, Organ. Sci..

[26]  G. Norman Likert scales, levels of measurement and the “laws” of statistics , 2010, Advances in health sciences education : theory and practice.

[27]  James N. Baron,et al.  Resources and Relationships: Social Networks and Mobility in the Workplace , 1997 .

[28]  Joshua Evan Blumenstock,et al.  Size matters: word count as a measure of quality on wikipedia , 2008, WWW.

[29]  M. Duijn,et al.  Software for social network analysis , 2005 .

[30]  Noah E. Friedkin,et al.  Information flow through strong and weak ties in intraorganizational social networks , 1982 .

[31]  Oded Nov,et al.  Information Quality in Wikipedia: The Effects of Group Composition and Task Conflict , 2011, J. Manag. Inf. Syst..

[32]  Mark S. Granovetter The Strength of Weak Ties , 1973, American Journal of Sociology.

[33]  Rocco J. Perla,et al.  Resolving the 50‐year debate around using and misusing Likert scales , 2008, Medical education.

[34]  Denilson Barbosa,et al.  Identifying controversial articles in Wikipedia: a comparative study , 2012, WikiSym '12.

[35]  Noah E. Friedkin,et al.  Theoretical Foundations for Centrality Measures , 1991, American Journal of Sociology.

[36]  Samer Faraj,et al.  Network Exchange Patterns in Online Communities , 2011, Organ. Sci..

[37]  Talel Abdessalem,et al.  Building a signed network from interactions in Wikipedia , 2011, DBSocial '11.

[38]  Starr Roxanne Hiltz,et al.  Structuring computer-mediated communication systems to avoid information overload , 1985, CACM.

[39]  G. Caldarelli,et al.  Taxonomy and clustering in collaborative systems: The case of the on-line encyclopedia Wikipedia , 2007, 0710.3058.

[40]  Param Vir Singh,et al.  A Hidden Markov Model of Developer Learning Dynamics in Open Source Software Projects , 2011, Inf. Syst. Res..

[41]  Sudha Ram,et al.  Who does what: Collaboration patterns in the wikipedia and their impact on article quality , 2011, TMIS.

[42]  Aniket Kittur,et al.  Effectiveness of shared leadership in online communities , 2012, CSCW.

[43]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[44]  Param Vir Singh,et al.  Developer Heterogeneity and Formation of Communication Networks in Open Source Software Projects , 2011, J. Manag. Inf. Syst..

[45]  Peter A. Gloor,et al.  Social capital increases efficiency of collaboration among Wikipedia editors , 2011, HT '11.

[46]  Carla O'Dell,et al.  If Only We Knew What We Know: Identification and Transfer of Internal Best Practices , 1998 .

[47]  Katherine Faust Centrality in affiliation networks , 1997 .

[48]  J. Powell,et al.  Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. , 2002, JAMA.

[49]  John Riedl,et al.  Tell me more: an actionable quality model for Wikipedia , 2013, OpenSym.

[50]  Morten T. Hansen,et al.  Competing for Attention in Knowledge Markets: Electronic Document Dissemination in a Management Consulting Company , 2001 .

[51]  Hyla H. Polen,et al.  Scope, Completeness, and Accuracy of Drug Information in Wikipedia , 2008, The Annals of pharmacotherapy.

[52]  Gary L. Lilien,et al.  Location, Location, Location: How Network Embeddedness Affects Project Success in Open Source Systems , 2006, Manag. Sci..

[53]  Gerald C. Kane,et al.  Network Characteristics and the Value of Collaborative User-Generated Content , 2011, Mark. Sci..

[54]  Daniel J. Brass,et al.  Network Analysis in the Social Sciences , 2009, Science.

[55]  D. Gergle,et al.  Hot Off the Wiki , 2013 .

[56]  Gerald C. Kane A multimethod study of information quality in wiki collaboration , 2011, TMIS.

[57]  Chaim Fershtman,et al.  Direct and indirect knowledge spillovers: the “social network” of open-source projects , 2011 .

[58]  Benno Stein,et al.  Identifying featured articles in wikipedia: writing style matters , 2010, WWW '10.