Who are the crowdworkers?: shifting demographics in mechanical turk

Amazon Mechanical Turk (MTurk) is a crowdsourcing system in which tasks are distributed to a population of thousands of anonymous workers for completion. This system is increasingly popular with researchers and developers. Here we extend previous studies of the demographics and usage behaviors of MTurk workers. We describe how the worker population has changed over time, shifting from a primarily moderate-income, U.S.-based workforce towards an increasingly international group with a significant population of young, well-educated Indian workers. This change in population points to how workers may treat Turking as a full-time job, which they rely on to make ends meet.

[1]  D. Gunzburg New forms of work organization , 1985 .

[2]  Uwe Hansmann,et al.  Pervasive Computing , 2003 .

[3]  Luis-Felipe Cabrera,et al.  AI Gets a Brain , 2006, ACM Queue.

[4]  S. Barley,et al.  Contracting: A New Form of Professional Practice , 2006 .

[5]  James Begole,et al.  An Exploration into Activity-Informed Physical Advertising Using PEST , 2007, Pervasive.

[6]  Brendan T. O'Connor,et al.  Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks , 2008, EMNLP.

[7]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[8]  Ed H. Chi,et al.  Towards a model of understanding social search , 2008, SSM '08.

[9]  Laura A. Dabbish,et al.  Designing games with a purpose , 2008, CACM.

[10]  Omar Alonso,et al.  Crowdsourcing for relevance evaluation , 2008, SIGF.

[11]  J. Zittrain,et al.  Ubiquitous human computing , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[12]  David A. Forsyth,et al.  Utility data annotation with Amazon Mechanical Turk , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[13]  Luis von Ahn Human computation , 2009, 2009 46th ACM/IEEE Design Automation Conference.

[14]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.