The Success and Failure of Quality Improvement Projects in Peer Production Communities

Peer production communities have been proven to be successful at creating valuable artefacts, with Wikipedia as a prime example. However, a number of studies have shown that work in these communities tends to be of uneven quality and certain content areas receive more attention than others. In this paper, we examine the efficacy of a range of targeted strategies to increase the quality of under-attended content areas in peer production communities. Mining data from five quality improvement projects in the English Wikipedia, the largest peer production community in the world, we show that certain types of strategies (e.g. creating artefacts from scratch) have better quality outcomes than others (e.g. improving existing artefacts), even if both are done by a similar cohort of participants. We discuss the implications of our findings for Wikipedia as well as other peer production communities.

[1]  Aniket Kittur,et al.  Coordination and beyond: social functions of groups in open content production , 2012, CSCW.

[2]  T. Yee.  The VGAM Package for Categorical Data Analysis , 2010 .

[3]  Aniket Kittur,et al.  Herding the cats: the influence of groups in coordinating peer production , 2009, Int. Sym. Wikis.

[4]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[5]  John Riedl,et al.  SuggestBot: using intelligent task routing to help people find work in wikipedia , 2007, IUI '07.

[6]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[7]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[8]  Joseph M. Reagle,et al.  Gender Bias in Wikipedia and Britannica , 2011 .

[9]  Jure Leskovec,et al.  Steering user behavior with badges , 2013, WWW.

[10]  Aaron Halfaker,et al.  Making peripheral participation legitimate: reader engagement experiments in wikipedia , 2013, CSCW.

[11]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[12]  Dan Cosley,et al.  What's mine is mine: territoriality in collaborative authoring , 2009, CHI.

[13]  M. Haklay.  How Good is Volunteered Geographical Information? A Comparative Study of OpenStreetMap and Ordnance Survey Datasets , 2010 .

[14]  Alexandre Ardichvili,et al.  Learning and Knowledge Sharing in Virtual Communities of Practice: Motivators, Barriers, and Enablers , 2008 .

[15]  Aniket Kittur,et al.  Effects of simultaneous and sequential work structures on distributed collaborative interdependent tasks , 2014, CHI.

[16]  John Riedl,et al.  Tell me more: an actionable quality model for Wikipedia , 2013, OpenSym.

[17]  Mikhil Masli,et al.  Leveraging the contributory potential of user feedback , 2014, CSCW.

[18]  Mikhil Masli,et al.  Eliciting and focusing geographic volunteer work , 2010, CSCW '10.

[19]  Alcides Velasquez,et al.  Classroom Wikipedia participation effects on future intentions to contribute , 2012, CSCW.

[20]  Robert E. Kraut,et al.  Wikipedia classroom experiment: bidirectional benefits of students' engagement in online production communities , 2013, CHI.

[21]  Aniket Kittur,et al.  Harnessing the wisdom of crowds in wikipedia: quality through coordination , 2008, CSCW.

[22]  Xin Li,et al.  Using Social Psychology to Motivate Contributions to Online Communities , 2005, J. Comput. Mediat. Commun..

[23]  D. Gergle,et al.  Hot Off the Wiki , 2013 .

[24]  Jacob Solomon,et al.  Bootstrapping wikis: developing critical mass in a fledgling community by seeding content , 2012, CSCW '12.

[25]  Les Gasser,et al.  Information quality work organization in wikipedia , 2008, J. Assoc. Inf. Sci. Technol..

[26]  Jenny Preece,et al.  Online Communities: Designing Usability and Supporting Sociability , 2000 .

[27]  M. Goodchild.  Citizens as sensors: the world of volunteered geography , 2007 .

[28]  A. Agresti,et al.  The analysis of ordered categorical data: An overview and a survey of recent developments , 2005 .

[29]  Kipling D. Williams,et al.  Interpersonal Relations and Group Processes Social Loafing: a Meta-analytic Review and Theoretical Integration , 2022 .

[30]  John Riedl,et al.  WP:clubhouse?: an exploration of Wikipedia's gender imbalance , 2011, Int. Sym. Wikis.

[31]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[32]  Monica Stephens.  Gender and the GeoWeb: divisions in the production of user-generated cartographic information , 2013, GeoJournal.

[33]  Jr. Frederick P. Brooks,et al.  The mythical man-month (anniversary ed.) , 1995 .

[34]  Robert E. Kraut,et al.  Mopping up: modeling wikipedia promotion decisions , 2008, CSCW.

[35]  Aaron Halfaker,et al.  A jury of your peers: quality, experience and ownership in Wikipedia , 2009, Int. Sym. Wikis.

[36]  John Riedl,et al.  Using intelligent task routing and contribution review to help communities build artifacts of lasting value , 2006, CHI.

[37]  Linda C. Smith,et al.  INFORMATION QUALITY IN A COMMUNITY-BASED ENCYCLOPEDIA , 2005 .

[38]  John Riedl,et al.  The effects of diversity on group productivity and member withdrawal in online volunteer groups , 2010, CHI.

[39]  Aaron Halfaker,et al.  Wikipedians are born, not made: a study of power editors on Wikipedia , 2009, GROUP.

[40]  Ivan Beschastnikh,et al.  Community, consensus, coercion, control: cs*w or how policy mediates mass participation , 2007, GROUP.

[41]  JapkowiczNathalie,et al.  The class imbalance problem: A systematic study , 2002 .

[42]  Aniket Kittur,et al.  Organizing without formal organization: group identification, goal setting and social modeling in directing online production , 2012, CSCW.

[43]  Les Gasser,et al.  Assessing Information Quality of a Community-Based Encyclopedia , 2005, ICIQ.

[44]  Robert E. Kraut,et al.  Motivating participation by displaying the value of contribution , 2006, CHI.

[45]  Dan Cosley,et al.  What do you know?: experts, novices and territoriality in collaborative systems , 2010, CHI.