Prolific.ac—A subject pool for online experiments

Abstract The number of online experiments conducted with subjects recruited via online platforms has grown considerably in the recent past. While one commercial crowdworking platform – Amazon’s Mechanical Turk – basically has established and since dominated this field, new alternatives offer services explicitly targeted at researchers. In this article, we present www.prolific.ac and lay out its suitability for recruiting subjects for social and economic science experiments. After briefly discussing key advantages and challenges of online experiments relative to lab experiments, we trace the platform’s historical development, present its features, and contrast them with requirements for different types of social and economic experiments.

[1]  Jonathan Robinson,et al.  TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences , 2016, Behavior Research Methods.

[2]  Jesse J. Chandler,et al.  Inside the Turk , 2014 .

[3]  U. Fischbacher,et al.  Lies in Disguise. An experimental study on cheating , 2013 .

[4]  M. Schraefel,et al.  “Now that you mention it”: A survey experiment on information, inattention and online privacy , 2017 .

[5]  Hyunji Kim,et al.  The Interrelations Between Social Class, Personal Relative Deprivation, and Prosociality , 2016, Social psychological and personality science.

[6]  Daniel Burton Shank,et al.  Using Crowdsourcing Websites for Sociological Research: The Case of Amazon Mechanical Turk , 2016 .

[7]  Thomas Stöckl,et al.  On the provision of incentives in finance experiments , 2018 .

[8]  David G. Rand,et al.  Economic Games on the Internet: The Effect of $1 Stakes , 2011, PloS one.

[9]  A. Acquisti,et al.  Beyond the Turk: Alternative Platforms for Crowdsourcing Behavioral Research , 2016 .

[10]  Antonio A. Arechar,et al.  Conducting interactive experiments online , 2017, Experimental Economics.

[11]  Ben Greiner,et al.  Subject pool recruitment procedures: organizing experiments with ORSEE , 2015, Journal of the Economic Science Association.

[12]  Stephanie Cacioppo,et al.  Measuring the Prevalence of Problematic Respondent Behaviors among MTurk, Campus, and Community Participants , 2016, PloS one.

[13]  Jesse Chandler,et al.  Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers , 2013, Behavior Research Methods.

[14]  John Bohannon,et al.  PSYCHOLOGY. Mechanical Turk upends social sciences. , 2016, Science.

[15]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[16]  David G. Rand,et al.  The promise of Mechanical Turk: how online labor markets can help theorists run behavioral experiments. , 2012, Journal of theoretical biology.

[17]  Duncan J. Watts,et al.  Cooperation and Contagion in Web-Based, Networked Public Goods Experiments , 2010, SECO.

[18]  Richard N. Landers,et al.  An Inconvenient Truth: Arbitrary Distinctions Between Organizational, Mechanical Turk, and Other Convenience Samples , 2015, Industrial and Organizational Psychology.

[19]  J. Yeluripati,et al.  Impact of Spatial Soil and Climate Input Data Aggregation on Regional Yield Simulations , 2016, PloS one.

[20]  David Huffman,et al.  Reference Points and Effort Provision , 2009, SSRN Electronic Journal.

[21]  David G. Rand,et al.  The online laboratory: conducting experiments in a real labor market , 2010, ArXiv.

[22]  Ben R. Newell,et al.  The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers , 2015, Judgment and Decision Making.

[23]  Charles Spence,et al.  ‘Show me the goods’: Assessing the effectiveness of transparent packaging vs. product imagery on product evaluation , 2018 .

[24]  Andreas Heinz,et al.  Changes in the Provision of Institutionalized Mental Health Care in Post-Communist Countries , 2012, PloS one.

[25]  D. Cooper.  A Note on Deception in Economic Experiments* , 2014, Journal of Wine Economics.

[26]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[27]  Todd M. Gureckis,et al.  Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research , 2013, PloS one.

[28]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[29]  Hans-Theo Normann,et al.  Experienced vs. inexperienced participants in the lab: do they behave differently? , 2017 .

[30]  Daniel L. Chen,et al.  oTree - An Open-Source Platform for Laboratory, Online, and Field Experiments , 2016 .

[31]  Andreas Nicklisch,et al.  hroot: Hamburg Registration and Organization Online Tool , 2014 .