Predicting Own Action: Self-Fulfilling Prophecy Induced by Proper Scoring Rules

This paper studies a mechanism to incentivize agents who predict their own future actions and truthfully declare their predictions. In a crowdsouring setting (e.g., participatory sensing), obtaining an accurate prediction of the actions of workers/agents is valuable for a requester who is collecting real-world information from the crowd. If an agent predicts an external event that she cannot control herself (e.g., tomorrow's weather), any proper scoring rule can give an accurate incentive. In our problem setting, an agent needs to predict her own action (e.g., what time tomorrow she will take a photo of a specific place) that she can control to maximize her utility. Also, her (gross) utility can vary based on an eternal event. We first prove that a mechanism can satisfy our goal if and only if it utilizes a strictly proper scoring rule, assuming that an agent can find an optimal declaration that maximizes her expected utility. This declaration is self-fulfilling; if she acts to maximize her utility, the probabilistic distribution of her action matches her declaration, assuming her prediction about the external event is correct. Furthermore, we develop a heuristic algorithm that efficiently finds a semi-optimal declaration, and show that this declaration is still self-fulfilling. We also examine our heuristic algorithm's performance and describe how an agent acts when she faces an unexpected scenario.

[1]  L. J. Savage.  Elicitation of Personal Probabilities and Expectations , 1971 .

[2]  L. V. Williams,et al.  Prediction Markets , 2003 .

[3]  David C. Parkes,et al.  A Robust Bayesian Truth Serum for Small Populations , 2012, AAAI.

[4]  Luis von Ahn.  Human Computation , 2008, ICDE.

[5]  Tuomas Sandholm,et al.  Decision rules and decision markets , 2010, AAMAS.

[6]  Manu Sridharan,et al.  Predicting your own effort , 2012, AAMAS.

[7]  Mausam,et al.  Crowdsourcing Control: Moving Beyond Multiple Choice , 2012, UAI.

[8]  Vincent Conitzer,et al.  Prediction Markets, Mechanism Design, and Cooperative Game Theory , 2009, UAI.

[9]  Ian A. Kash,et al.  Market manipulation with outside incentives , 2011, Autonomous Agents and Multi-Agent Systems.

[10]  Makoto Yokoo,et al.  Ability Grouping of Crowd Workers via Reward Discrimination , 2013, HCOMP.

[11]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[12]  David M. Pennock,et al.  Designing Markets for Prediction , 2010, AI Mag..

[13]  Craig Boutilier,et al.  Eliciting forecasts from self-interested experts: scoring rules for decision makers , 2011, AAMAS.

[14]  Vincent Conitzer,et al.  Prediction Mechanisms That Do Not Incentivize Undesirable Actions , 2009, WINE.

[15]  Deborah Estrin,et al.  Recruitment Framework for Participatory Sensing Data Collections , 2010, Pervasive.

[16]  R. L. Winkler,et al.  Scoring Rules for Continuous Probability Distributions , 1976 .

[17]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[18]  Ian A. Kash,et al.  Information elicitation for decision making , 2011, AAMAS.

[19]  Ruggiero Cavallo,et al.  Winner-Take-All Crowdsourcing Contests with Stochastic Production , 2013, HCOMP.