How Online Labor Markets are Revolutionizing Innovation and Discovery in the Social Sciences

The internet provides an unprecedented opportunity for social scientists to recruit large number of subjects quickly, cheaply and virtually effortlessly. Online labor markets, such as Amazon Mechanical Turk (MTurk), allow researchers to easily recruit and pay subjects from around the world to participate in studies which are monetarily incentivized (ie pay depends on choices in the study, rather just a flat rate). These labor markets also facilitate field studies, where 'subjects' are unaware they are in an experiment, but instead think they are just completing normal work tasks. The speed and easy of online experimentation has the potential to increase the rate of scientific progress by orders of magnitude. In this talk I will describe how MTurk can be used to run incentivized experiments, a number of validation studies demonstrating the reliability on data collected on MTurk, and new insights into human behavior emerging from MTurk experiments.