The Crowdsourcing Design Space

Crowdsourcing is a new kind of organizational structure, one that is conducive to large amounts of short parallel work: thousands of individuals may work for several minutes on tasks, their outputs aggregated into a useful product or service. The dimensions of this new organizational form are described. Areas for future research are identified, focusing on open-ended tasks and the coordination structures that might foster collective creativity.

[1]  Karim R. Lakhani,et al.  Parallel Search , Incentives and Problem Type : Revisiting the Competition and Innovation Link Parallel Search , Incentives and Problem Type : Revisiting the Competition and Innovation Link 1 , 2008 .

[2]  Robin Milner,et al.  Communicating and mobile systems - the Pi-calculus , 1999 .

[3]  W. Estes,et al.  Risks of drawing inferences about cognitive processes from model fits to individual versus average performance , 2005, Psychonomic bulletin & review.

[4]  Joel West,et al.  How open is open enough?: Melding proprietary and open source platform strategies , 2003 .

[5]  Ned Gulley.  Patterns of innovation: a web-based MATLAB programming contest , 2001, CHI Extended Abstracts.

[6]  Jim Everett,et al.  Coming of Age in Second Life: An Anthropologist Explores the Virtually Human , 2010 .

[7]  Jeffrey E. Boyd,et al.  SwarmArt: interactive art from swarm intelligence , 2004, MULTIMEDIA '04.

[8]  J. Gero.  Computational Models of Innovative and Creative Design Processes , 2000 .

[9]  Jeffrey V. Nickerson,et al.  Appropriation and Creativity: User-Initiated Contests in Scratch , 2011, 2011 44th Hawaii International Conference on System Sciences.

[10]  Ted S. Sindlinger,et al.  Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business , 2010 .

[11]  Laura A. Dabbish,et al.  Designing games with a purpose , 2008, CACM.

[12]  Kevin Crowston,et al.  The interdisciplinary study of coordination , 1994, CSUR.

[13]  Yehuda Koren,et al.  Lessons from the Netflix prize challenge , 2007, SKDD.

[14]  Jeffrey V. Nickerson,et al.  Cooks or cobblers?: crowd creativity through combination , 2011, CHI.

[15]  Bruno Galantucci,et al.  An Experimental Study of the Emergence of Human Communication Systems , 2005, Cogn. Sci..

[16]  John Voiklis,et al.  A Thing Is What We Say It Is: Referential Communication and Indirect Category Learning , 2008 .

[17]  B. Nijstad,et al.  Group creativity : An introduction , 2003 .

[18]  Robert L. Goldstone,et al.  Propagation of innovations in networked groups. , 2008, Journal of experimental psychology. General.

[19]  Alan R. Dennis,et al.  Electronic brainstorming: Theory, research, and future directions. , 2003 .

[20]  Aniket Kittur,et al.  Crowdsourcing, collaboration and creativity , 2010, XRDS.

[21]  F. Galton.  Vox Populi , 1907, Nature.

[22]  John H. Holland,et al.  Building Blocks, Cohort Genetic Algorithms, and Hyperplane-Defined Functions , 2000, Evolutionary Computation.

[23]  Eric Rosenbaum,et al.  Scratch: programming for all , 2009, Commun. ACM.

[24]  Jeffrey V. Nickerson,et al.  Crowdsourcing Creativity: Combining Ideas in Networks , 2010 .

[25]  Frederick P. Brooks,et al.  The Design of Design: Essays from a Computer Scientist , 2010 .

[26]  Daren C. Brabham.  MOVING THE CROWD AT THREADLESS , 2010 .

[27]  H. Gurnee.  Maze Learning in the Collective Situation , 1937 .

[28]  Patrick Valduriez,et al.  Distributed and parallel database systems , 1996, CSUR.

[29]  Alex Kosorukoff,et al.  Human based genetic algorithm , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[30]  Alexander S. Szalay,et al.  Galaxy Zoo: An Experiment in Public Science Participation , 2007 .

[31]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[32]  Kevin Crowston,et al.  ICIS 2008 Proceedings , 2008 .

[33]  Jay F. Nunamaker,et al.  Electronic meeting systems , 1991, CACM.

[34]  Alex Pentland,et al.  Time-Critical Social Mobilization , 2010, Science.

[35]  Pamela J. Hinds,et al.  Distributed Work , 2002 .

[36]  M. Lorenzen.  Creativity in context : Content, cost, chance and collection in the organization of the film industry , 2009 .

[37]  Sharon Bailin.  CREATIVITY IN CONTEXT , 2002 .

[38]  E. Deci,et al.  A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. , 1999, Psychological bulletin.

[39]  Matthew J. Salganik,et al.  Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market , 2006, Science.

[40]  Benjamin B. Bederson,et al.  Human computation: a survey and taxonomy of a growing field , 2011, CHI.

[41]  M. Kearns,et al.  An Experimental Study of the Coloring Problem on Human Subject Networks , 2006, Science.

[42]  Lydia B. Chilton,et al.  Exploring iterative and parallel human computation processes , 2010, HCOMP '10.

[43]  Jock Given,et al.  The wealth of networks: How social production transforms markets and freedom , 2007, Inf. Econ. Policy.

[44]  Jeffrey V. Nickerson,et al.  The Spatial Nature of Thought: Understanding Systems Design Through Diagrams , 2008, ICIS.

[45]  Marilyn Tremaine.  CHI '01 Extended Abstracts on Human Factors in Computing Systems , 2001, CHI Extended Abstracts.

[46]  Wil M. P. van der Aalst,et al.  Workflow Patterns , 2003, Distributed and Parallel Databases.

[47]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[48]  Chrysanthos Dellarocas,et al.  Harnessing Crowds: Mapping the Genome of Collective Intelligence , 2009 .

[49]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[50]  Radhika Jain.  Investigation of Governance Mechanisms for Crowdsourcing Initiatives , 2010, AMCIS.