A Game-Theoretic Analysis of the ESP Game ∗ PRELIMINARY DRAFT : COMMENTS WELCOME

In recent years, there has been a great deal of progress in “Games with a Purpose,” interactive games that users play because they are fun, with the added benefit that they are doing useful work in the process. The ESP game, developed by von Ahn and Dabbish [24], is an example of such a game devised to label images on the web. Since labeling images is a hard problem for computer vision algorithms and can be tedious and time-consuming for humans, the ESP game provides humans with incentive to do useful work by being enjoyable to play. We present a simple game-theoretic model of the ESP game and characterize the equilibrium behavior in our model. Our equilibrium analysis supports the fact that users appear to be coordinating on low effort words. We provide an alternate model of utility and show that equilibrium behavior in this model achieves more desirable outcomes, from the system designer’s perspective. We also give sufficient conditions for coordinating on high effort words to be a BayesianNash equilibrium. Our results suggest the possibility of formal incentive design in achieving desirable system-wide outcomes in this area of “human computation” in complementing existing considerations of robustness against cheating and human factors.

[1]  Jane Yung-jen Hsu,et al.  KissKissBan: a competitive human computation game for image annotation , 2010, HCOMP '09.

[2]  Milan Vojnovic,et al.  Crowdsourcing and all-pay auctions , 2009, EC '09.

[3]  David C. Parkes,et al.  Designing incentives for online question and answer forums , 2009, EC '09.

[4]  David C. Parkes,et al.  Policy teaching through reward function learning , 2009, EC '09.

[5]  Kuan-Ta Chen,et al.  On formal models for social verification , 2009, HCOMP '09.

[6]  David C. Parkes,et al.  The role of game theory in human computation systems , 2009, HCOMP '09.

[7]  Stephen E. Robertson,et al.  Rethinking the ESP game , 2009, CHI Extended Abstracts.

[8]  Luis von Ahn,et al.  Matchin: eliciting user preferences with an online game , 2009, CHI.

[9]  Mark S. Ackerman,et al.  Questions in, knowledge in?: a study of naver's question answering community , 2009, CHI.

[10]  Laura A. Dabbish,et al.  Designing games with a purpose , 2008, CACM.

[11]  Mark S. Ackerman,et al.  Crowdsourcing and knowledge sharing: strategic user behavior on taskcn , 2008, EC '08.

[12]  Avi Pfeffer,et al.  Modeling how humans reason about others with partial information , 2008, AAMAS.

[13]  Jane Yung-jen Hsu,et al.  PhotoSlap: A Multi-player Online Game for Semantic Annotation , 2007, AAAI.

[14]  Jane Yung-jen Hsu,et al.  The PhotoSlap Game: Play to Annotate , 2007, AAAI.

[15]  M. Kearns,et al.  An Experimental Study of the Coloring Problem on Human Subject Networks , 2006, Science.

[16]  Luis von Ahn Games with a Purpose , 2006, Computer.

[17]  Manuel Blum,et al.  Verbosity: a game for collecting common-sense facts , 2006, CHI.

[18]  Manuel Blum,et al.  Improving accessibility of the web with a computer game , 2006, CHI.

[19]  Manuel Blum,et al.  Peekaboom: a game for locating objects in images , 2006, CHI.

[20]  T. Mulgan The Contract Theory , 2006 .

[21]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[22]  Dipjyoti Majumdar,et al.  Ordinally Bayesian Incentive Compatible Stable Matching , 2003 .

[23]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[24]  Y. Benkler Coase's Penguin, or Linux and the Nature of the Firm , 2001, ArXiv.

[25]  H. Levy,et al.  Efficiency analysis of choices involving risk , 1969 .

[26]  C. Papadimitriou,et al.  A Game-Theoretic Analysis of Games with a Purpose , 2008 .

[27]  C. d'Aspremont,et al.  Ordinal Bayesian incentive compatible representations of committees , 1988 .

[28]  David C. Parkes,et al.  Value-Based Policy Teaching with Active Indirect Elicitation , 2008, AAAI.