Comparisons of Online Recruitment Strategies for Convenience Samples

The rise of social media websites (e.g., Facebook) and online services such as Google AdWords and Amazon Mechanical Turk (MTurk) offers new opportunities for researchers to recruit study participants. Although researchers have started to use these emerging methods, little is known about how they perform in terms of cost efficiency and, more importantly, the types of people that they ultimately recruit. Here, we report findings about the performance of four online sources for recruiting iPhone users to participate in a web survey. The findings reveal very different performances between two types of strategies: those that “pull in” online users actively looking for paid work (MTurk workers and Craigslist users) and those that “push out” a recruiting ad to online users engaged in other, unrelated online activities (Google AdWords and Facebook). The pull-method recruits were more cost efficient and committed to the survey task, while the push-method recruits were more demographically diverse.

[1]  David G. Rand,et al.  The online laboratory: conducting experiments in a real labor market , 2010, ArXiv.

[2]  Todd M. Gureckis,et al.  Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research , 2013, PloS one.

[3]  Tara S. Behrend,et al.  The viability of crowdsourcing for survey research , 2011, Behavior research methods.

[4]  S. Boles,et al.  Successful participant recruitment strategies for an online smokeless tobacco cessation program. , 2006, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[5]  J. Krosnick Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[6]  Danielle E. Ramo,et al.  Reaching young adult smokers through the internet: comparison of three recruitment mechanisms. , 2010, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[7]  Thomas R. Ioerger,et al.  Precision and Disclosure in Text and Voice Interviews on Smartphones , 2015, PloS one.

[8]  Nathan K. Cobb,et al.  Characteristics of smokers reached and recruited to an internet smoking cessation trial: a case of denominators. , 2006, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[9]  Danielle E. Ramo,et al.  Broad Reach and Targeted Recruitment Using Facebook for an Online Survey of Young Adult Substance Use , 2012, Journal of medical Internet research.

[10]  E. Singer,et al.  Leverage-saliency theory of survey participation: description and an illustration. , 2000, Public opinion quarterly.

[11]  A. Mackinnon,et al.  Internet-Based Recruitment to a Depression Prevention Intervention: Lessons From the Mood Memos Study , 2013, Journal of medical Internet research.

[12]  Adam J. Berinsky,et al.  Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk , 2012, Political Analysis.

[13]  D. Yeager,et al.  Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples , 2011 .

[14]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[15]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[16]  Roger Tourangeau,et al.  Summary Report of the AAPOR Task Force on Non-probability Sampling , 2013 .