How different types of participant payments alter task performance

Researchers typically use incentives (such as money or course credit) in order to obtain participants who engage in the specific behaviors of interest to the researcher. There is, however, little understanding or agreement on the effects of different types and levels of incentives used. Some results in the domain of statistical reasoning suggest that performance differences --- previously deemed theoretically important --- may actually be due to differences in incentive types across studies. 704 participants completed one of five variants of a statistical reasoning task, for which they received either course credit, flat fee payment, or performance-based payment incentives. Successful task completion was more frequent with performance-based incentives than with either of the other incentive types. Performance on moderately difficult tasks (compared to very easy and very hard tasks) was most sensitive to incentives. These results can help resolve existing debates about inconsistent findings, guide more accurate comparisons across studies, and be applied beyond research settings.

[1]  L. Crespi.  Quantitative variation of incentive and performance in the white rat. , 1942 .

[2]  L. Crespi.  Amount of reinforcement and level of performance. , 1944 .

[3]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[4]  Robert D. Pritchard,et al.  The influence of goal setting and financial incentives on task performance , 1973 .

[5]  William F. Wright,et al.  Effects of situation familiarity and financial incentives on use of the anchoring and adjustment heuristic for probability assessment , 1989 .

[6]  William D. Crano,et al.  Pitfalls Associated With the Use of Financial Incentives (and Other Complex Manipulations) in Human Social Research , 1991 .

[7]  Gerd Gigerenzer,et al.  How to Improve Bayesian Reasoning Without Instruction: Frequency Formats , 1995 .

[8]  D. Ziebart,et al.  A Model of Financial Incentive Effects in Decision Making , 1995 .

[9]  L. Cosmides,et al.  Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty , 1996, Cognition.

[10]  J. Shaw,et al.  Are financial incentives related to performance? A meta-analytic review of empirical research. , 1998 .

[11]  Gary L. Brase,et al.  Individuation, counting, and statistical inference: The role of frequency and whole-object representations in judgment under uncertainty , 1998 .

[12]  Colin Camerer,et al.  The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework , 1999 .

[13]  Peter Sedlmeier,et al.  Improving Statistical Reasoning: Theoretical Models and Practical Implications , 1999 .

[14]  Jonathan Evans,et al.  Frequency versus probability formats in statistical word problems , 2000, Cognition.

[15]  Gerd Gigerenzer,et al.  Communicating Statistical Information , 2000, Science.

[16]  R. Hertwig,et al.  Experimental practices in economics: A methodological challenge for psychologists? , 2001, Behavioral and Brain Sciences.

[17]  V. Girotto,et al.  Solving probabilistic and statistical problems: a matter of information structure and question form , 2001, Cognition.

[18]  D. Shanks,et al.  A Re-examination of Probability Matching and Rational Choice , 2002 .

[19]  Gary L. Brase,et al.  Ecological and evolutionary validity: comments on Johnson-Laird, Legrenzi, Girotto, Legrenzi, and Caverni's (1999) mental-model theory of extensional reasoning. , 2002, Psychological review.

[20]  Amnon Rapoport,et al.  Effects of Financial Incentives on the Breakdown of Mutual Trust , 2002, Psychological science.

[21]  Jeffrey M. Stibel,et al.  Frequency illusions and other fallacies , 2003 .

[22]  Andreas Ortmann,et al.  How financial incentives and cognitive abilities affect task performance in laboratory settings: an illustration † , 2004 .

[23]  N. Epley,et al.  When effortful thinking influences judgmental anchoring: differential effects of forewarning and incentives on self‐generated and externally provided anchors , 2005 .

[24]  Gary L. Brase,et al.  Participant recruitment methods and statistical reasoning performance , 2006, Quarterly journal of experimental psychology.

[25]  Miranda R. Goode,et al.  The Psychological Consequences of Money , 2006, Science.

[26]  Benjamin Y Hayden,et al.  Economic principles motivating social attention in humans , 2007, Proceedings of the Royal Society B: Biological Sciences.

[27]  Ashwani Monga,et al.  How I Decide Depends on What I Spend: Use of Heuristics is Greater for Time than for Money , 2007 .

[28]  Gary L. Brase,et al.  Frequency interpretation of ambiguous statistical information facilitates Bayesian reasoning , 2008, Psychonomic bulletin & review.

[29]  Gary L. Brase.  Pictorial representations in statistical reasoning , 2009 .