Influencing and Measuring Behaviour in Crowdsourced Activities

Crowdsourcing psychometric data is common in areas of Human-Computer Interaction (HCI) such as information visualization, text entry, and interface design. In some of the social sciences, crowdsourcing data is now considered routine, and even standard. In this chapter, we explore the collection of data in this manner, beginning by describing the variety of approaches can be used to crowdsource data. Then, we evaluate past literature that has compared the results of these approaches to more traditional data-collection techniques. From this literature, we synthesize a set of design and implementation guidelines for crowdsourcing studies. Finally, we describe how particular analytic techniques can be recruited to aid the analysis of large-scale crowdsourced data. The goal of this chapter it to clearly enumerate the difficulties of crowdsourcing psychometric data and to explore how, with careful planning and execution, these limitations can be overcome.