Crowdsourcing Government: Lessons from Multiple Disciplines

Crowdsourcing has proliferated across disciplines and professional fields. Implementers in the public sector face practical challenges, however, in the execution of crowdsourcing. This review synthesizes prior crowdsourcing research and practices from a variety of disciplines and focuses to identify lessons for meeting the practical challenges of crowdsourcing in the public sector. It identifies three distinct categories of crowdsourcing: organizations, products and services, and holistic systems. Lessons about the fundamental logic of process design—alignment, motivation, and evaluation—identified across the three categories are discussed. Conclusions drawn from past studies and the resulting evidence can help public managers better design and implement crowdsourcing in the public sector.

[1]  Marco Zamarian,et al.  Analyzing Crowd Labor and Designing Incentives for Humans in the Loop , 2012, IEEE Internet Computing.

[2]  Brendan T. O'Connor,et al.  Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks , 2008, EMNLP.

[3]  Dimitra Anastasiou,et al.  Comparison of crowdsourcing translation with Machine Translation , 2011, J. Inf. Sci..

[4]  S. Ogawa,et al.  User-Generated Versus Designer-Generated Products: A Performance Assessment at Muji , 2012 .

[5]  Christopher L. Tucci,et al.  Crowdsourcing As a Solution to Distant Search , 2012 .

[6]  Clifford A. Lampe,et al.  Ratings Use in an Online Discussion System: The Slashdot Case , 2006 .

[7]  John Riedl,et al.  Crowdsourcing medical research , 2013, Computer.

[8]  Linus Dahlander,et al.  Distant Search, Narrow Attention: How Crowding Alters Organizations’ Filtering of Suggestions in Crowdsourcing , 2014 .

[9]  E. Seltzer,et al.  Citizen Participation, Open Innovation, and Crowdsourcing , 2013 .

[10]  Ann Light,et al.  Toward a Typology of Participation in Crowdwork , 2016, CSCW Companion.

[11]  Pär J. Ågerfalk,et al.  Outsourcing to an Unknown Workforce: Exploring Opensourcing as a Global Sourcing Strategy , 2008, MIS Q..

[12]  Archon Fung Putting the Public Back into Governance: The Challenges of Citizen Participation and Its Future , 2015 .

[13]  Daren C. Brabham Crowdsourcing in the Public Sector , 2015 .

[14]  Carl F. Salk,et al.  Comparing the Quality of Crowdsourced Data Contributed by Expert and Non-Experts , 2013, PloS one.

[15]  Souad Djelassi,et al.  Customers' participation in product development through crowdsourcing: Issues and implications , 2013 .

[16]  Atreyi Kankanhalli,et al.  Investigating the antecedents of organizational task crowdsourcing , 2015, Inf. Manag..

[17]  Dana Chandler,et al.  Breaking Monotony with Meaning: Motivation in Crowdsourcing Markets , 2012, ArXiv.

[18]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[19]  Taewoo Nam,et al.  Suggesting frameworks of citizen-sourcing via Government 2.0 , 2012, Gov. Inf. Q..

[20]  Gavin Newsom,et al.  Citizenville: How to Take the Town Square Digital and Reinvent Government , 2013 .

[21]  H. George Frederickson and the Dialogue on Citizenship in Public Administration , 2012 .

[22]  Qinghua Zhu,et al.  Effects of extrinsic and intrinsic motivation on participation in crowdsourcing contest: A perspective of self-determination theory , 2014, Online Inf. Rev..

[23]  D. Clery Galaxy evolution. Galaxy zoo volunteers share pain and glory of research. , 2011, Science.

[24]  Joseph Lampel,et al.  Test-Driving the Future: How Design Competitions Are Changing Innovation , 2012 .

[25]  K. Desouza,et al.  Implementing Open Innovation in the Public Sector : the Case of Challenge.gov , 2013 .

[26]  Jinlong Zhang,et al.  Perceived justice and creativity in crowdsourcing communities: Empirical evidence from China , 2015 .

[27]  Xu-Dong Zhang,et al.  Learning to Rank from Noisy Data , 2015, ACM Trans. Intell. Syst. Technol..

[28]  Ines Mergel,et al.  Toward Open Public Administration Scholarship , 2011 .

[29]  Barry L. Bayus,et al.  Crowdsourcing New Product Ideas over Time: An Analysis of the Dell IdeaStorm Community , 2013, Manag. Sci..

[30]  Todd M. Gureckis,et al.  Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research , 2013, PloS one.

[31]  J. Chataway,et al.  Crowdsourcing Based Business Models , 2012 .

[32]  Sally Ellis,et al.  A History of Collaboration, a Future in Crowdsourcing: Positive Impacts of Cooperation on British Librarianship , 2014 .

[33]  Leslie Brokaw,et al.  Does sustainability change the talent equation , 2009 .

[34]  Daniel J. Veit,et al.  More than fun and money. Worker Motivation in Crowdsourcing - A Study on Mechanical Turk , 2011, AMCIS.

[35]  David A. Askay A Conceptual Framework for Investigating Organizational Control and Resistance in Crowd-Based Platforms , 2017, HICSS.

[36]  J. C. Thomas Citizen, Customer, Partner: Rethinking the Place of the Public in Public Management , 2013 .

[37]  Lisa Blomgren Amsler,et al.  Direct Public Engagement in Local Government , 2014 .

[38]  Patrice Dutil Crowdsourcing as a new instrument in the government's arsenal: Explorations and considerations , 2015 .

[39]  Fernando González-Ladrón-de-Guevara,et al.  Towards an integrated crowdsourcing definition , 2012, J. Inf. Sci..

[40]  Jeffrey L. Brudney,et al.  Coproduction of Government Services and the New Information Technology: Investigating the Distributional Biases , 2013 .

[41]  R. Lutz Marketing Scholarship 2.0 , 2011 .

[42]  Chris Callison-Burch,et al.  Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon’s Mechanical Turk , 2009, EMNLP.

[43]  C. Knox Public Administrators’ Use of Social Media Platforms , 2016 .

[44]  Ann Majchrzak,et al.  Towards an Information Systems Perspective and Research Agenda on Crowdsourcing for Innovation , 2013, J. Strateg. Inf. Syst..

[45]  Cliff Lampe,et al.  Crowdsourcing civility: A natural experiment examining the effects of distributed moderation in online forums , 2014, Gov. Inf. Q..

[46]  Peter deLeon,et al.  Democracy and the policy sciences , 1997 .

[47]  François Charoy,et al.  CrowdSC: Building Smart Cities with Large-Scale Citizen Participation , 2013, IEEE Internet Computing.

[48]  Christina Demski,et al.  Creating a national citizen engagement process for energy policy , 2014, Proceedings of the National Academy of Sciences.

[49]  Lisa Schmidthuber,et al.  Unleashing Innovation beyond Organizational Boundaries: Exploring Citizensourcing Projects , 2018 .

[50]  Yun Wan,et al.  The Matthew Effect in social commerce , 2015, Electron. Mark..

[51]  Devavrat Shah,et al.  Budget-Optimal Task Allocation for Reliable Crowdsourcing Systems , 2011, Oper. Res..

[52]  Paul T. Jaeger,et al.  The impact of polices on government social media usage: Issues, challenges, and recommendations , 2012, Gov. Inf. Q..

[53]  Peter A. Johnson,et al.  Civic Hackathons: New Terrain for Local Government-Citizen Interaction? , 2016 .

[54]  Rob Kling,et al.  a Bit More to It: Scholarly Communication Forums as Socio-technical Interaction Networks , 2003, J. Assoc. Inf. Sci. Technol..

[55]  Chris Mellish,et al.  Crowdsourcing Without a Crowd , 2016, ACM Trans. Intell. Syst. Technol..

[56]  Helen K. Liu Exploring Online Engagement in Public Policy Consultation: The Crowd or the Few? , 2017 .

[57]  Jan Marco Leimeister,et al.  Crowdsourcing: How to Benefit from (Too) Many Great Ideas , 2013, MIS Q. Executive.

[58]  Sonja Marjanovic,et al.  Crowdsourcing based business models: In search of evidence for innovation 2.0 , 2012 .

[59]  Tanja Aitamurto,et al.  Five Design Principles for Crowdsourced Policymaking: Assessing the Case of Crowdsourced Off-Road Traffic Law in Finland , 2015 .

[60]  Jie Zhang,et al.  Task Division for Team Success in Crowdsourcing Contests: Resource Allocation and Alignment Effects , 2015, J. Manag. Inf. Syst..

[61]  Kurt Matzler,et al.  Communitition: The Tension between Competition and Collaboration in Community-Based Design Contests , 2011 .

[62]  Tanja Aitamurto,et al.  Crowdsourcing for Democracy: A New Era in Policy-Making , 2012 .

[63]  Fang Wu,et al.  Crowdsourcing, attention and productivity , 2008, J. Inf. Sci..

[64]  Kevin J. Boudreau,et al.  Let a Thousand Flowers Bloom? An Early Look at Large Numbers of Software 'Apps' Developers and Patterns of Innovation , 2011, Organ. Sci..

[65]  Nikolaus Franke,et al.  "Does This Sound Like a Fair Deal?": Antecedents and Consequences of Fairness Expectations in the Individual's Decision to Participate in Firm Innovation , 2013, Organ. Sci..

[66]  Scott R. B. Minkoff,et al.  NYC 311 , 2016 .

[67]  Nicholas C. Zingale,et al.  A Framework for Using Crowdsourcing in Government , 2016, Social Entrepreneurship.

[68]  Martin Schreier,et al.  The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas? , 2009 .

[69]  Tara S. Behrend,et al.  The viability of crowdsourcing for survey research , 2011, Behavior research methods.

[70]  Rajiv Kishore,et al.  Rules of Crowdsourcing: Models, Issues, and Systems of Control , 2013, Inf. Syst. Manag..

[71]  Tuomas Ahola,et al.  A network perspective on idea and innovation crowdsourcing in industrial firms , 2014 .

[72]  Elisa Bertino,et al.  Representation and querying of unfair evaluations in social rating systems , 2014, Comput. Secur..

[73]  Rashid Mehmood,et al.  The strategic prototype crime-sourcing and the science/science fiction behind it , 2014 .

[74]  Daren C. Brabham Motivations for Participation in a Crowdsourcing Application to Improve Public Engagement in Transit Planning , 2012 .

[75]  Dennis Linders,et al.  From e-government to we-government: Defining a typology for citizen coproduction in the age of social media , 2012, Gov. Inf. Q..

[76]  Yongqiang Sun,et al.  Investigating the Non-Linear Relationships in the Expectancy Theory: The Case of Crowdsourcing Marketplace , 2012, AMCIS.

[77]  Ines Mergel Opening Government , 2015 .

[78]  L. Jeppesen,et al.  The Value of Openness in Scientific Problem Solving , 2007 .

[79]  B. Noveck Wiki Government: How Technology Can Make Government Better, Democracy Stronger, and Citizens More Powerful , 2009 .

[80]  Karim R Lakhani,et al.  Using the crowd as an innovation partner. , 2013, Harvard business review.

[81]  MellishChris,et al.  Crowdsourcing Without a Crowd , 2016 .

[82]  Fabio Nonino,et al.  Open innovation web-based platforms: The impact of different forms of motivation on collaboration , 2012 .

[83]  J. Dewey,et al.  The Public and its Problems , 1927 .

[84]  E. Bonabeau Decisions 2.0: the power of collective intelligence , 2009 .

[85]  John M. Bryson,et al.  Designing Public Participation Processes , 2015 .

[86]  Mary J. Newhart,et al.  Not by technology alone: The "analog" aspects of online public engagement in policymaking , 2014, Gov. Inf. Q..

[87]  A. Majchrzak,et al.  Open Innovation Requires Integrated Competition-Community Ecosystems: Lessons Learned from Civic Open Innovation , 2014 .

[88]  Steve Baron,et al.  Individual Customers? Use and Integration of Resources: Empirical Findings and Organizational Implications in the Context of Value Co-creation , 2011 .

[89]  Chiara Franzoni,et al.  Crowd Science: The Organization of Scientific Research in Open Collaborative Projects , 2014 .

[90]  Adriana Santarosa Vivacqua,et al.  Enabling Crowd Participation in Governmental Decision-making , 2011, J. Univers. Comput. Sci..

[91]  Marian Garcia Martinez,et al.  The wisdom of crowds: The potential of online communities as a tool for data analysis , 2014 .

[92]  Eric T. Meyer,et al.  Examining the Hyphen: The Value of Social Informatics for Research and Teaching , 2014 .

[93]  R. Kozinets,et al.  The Wisdom of Consumer Crowds , 2008 .

[94]  Josh Lerner,et al.  The Simple Economics of Open Source , 2000 .

[95]  Ines Mergel,et al.  A Three‐Stage Adoption Process for Social Media Use in Government , 2013 .

[96]  Araz Taeihagh,et al.  The Fundamentals of Policy Crowdsourcing , 2015, ArXiv.