Reward Systems in Human Computation Games

Human computation games (HCGs) are games in which player interaction is used to solve problems intractable for computers. Most HCGs use simple reward mechanisms such as points or leaderboards, but in contrast, many mainstream games use more complex, and often multiple, reward mechanisms. In this paper, we investigate whether multiple reward systems and ability to choose the type of reward affects human task performance and player experience in HCGs. We conducted a study using a cooking-themed HCG, Cafe Flour Sack, which implements four reward systems, and had two experimental versions: one which randomly assigns rewards and the other which offers players the choice of reward. Players were recruited from both Amazon Mechanical Turk and university students. We report the results across these different game versions and player audiences. Our results suggest that offering players a choice of reward can yield better \emph{task completion} metrics and similarly-engaged \emph{player experiences}, and may improve these metrics and experiences for audiences that are not experts in crowdsourcing. We discuss these and other results in the broader context of exploring different rewards systems and other aspects of reward mechanics in HCGs.