G T ] 3 M ay 2 01 6 A Framework For Designing Information Elicitation Mechanisms That Reward Truth-telling Yuqing Kong University of Michigan

Information elicitation mechanisms, such as Peer Prediction [Miller 2005] and Bayesian Truth Serum [Prelec 2004], are designed to reward agents for honestly reporting their private information, even when this information cannot be directly verified. Information elicitation mechanisms, such as these, are cleverly designed so that truth-telling is a strict Bayesian Nash Equilibrium. However, a key challenge that has arisen is that there are typically many other non-truthful equilibrium as well, and it is important that truth-telling not only be an equilibrium, but be paid more than other equilibrium so that agents do not coordinate on a non-informative equilibria. Several past works have overcome this challenge in various setting using clever techniques, but a new technique was required for each new setting. Our main contribution in this paper is providing a framework for designing information elicitation mechanisms where truth-telling is the highest paid equilibrium, even when the mechanism does not know the common prior. We define information monotone functions which can measure the amount of "information" contained in agents' reports such that the function is greater in the truthful equilibrium than in non-truthful equilibria. We provide several interesting information monotone functions ( f -disagreement, f-mutual information, f -information gain) in different settings. Aided by these theoretical tools, we (1) extend Dasgupta and Ghosh[2013]'s mechanism to the non-binary setting with an additional assumption that agents are asked to answer a large number of a priori similar questions; (2) reprove the main results of Prelec[2004], Dasgupta and Ghosh[2013] and a weaker version of Kong and Schoenebeck[2016] in our framework. Our framework thus provides both new mechanisms and a deeper understanding of prior results.

[1]  Boi Faltings,et al.  Robust Incentive-Compatible Feedback Payments , 2006, TADA/AMEC.

[2]  Boi Faltings,et al.  Incentives for Truthful Information Elicitation of Continuous Signals , 2014, AAAI.

[3]  Blake Riley,et al.  Minimum Truth Serums with Optional Predictions , 2014 .

[4]  Boi Faltings,et al.  A Robust Bayesian Truth Serum for Non-Binary Signals , 2013, AAAI.

[5]  S. Amari,et al.  Information geometry of divergence functions , 2010 .

[6]  Grant Schoenebeck,et al.  Putting Peer Prediction Under the Micro(economic)scope and Making Truth-Telling Focal , 2016, WINE.

[7]  Grant Schoenebeck,et al.  Equilibrium Selection in Information Elicitation without Verification via Information Monotonicity , 2018, ITCS.

[8]  Yiling Chen,et al.  Elicitability and knowledge-free elicitation with peer prediction , 2014, AAMAS.

[9]  Boi Faltings,et al.  Incentives to Counter Bias in Human Computation , 2014, HCOMP.

[10]  Arpit Agarwal,et al.  Informed Truthfulness in Multi-Task Peer Prediction , 2016, EC.

[11]  Boi Faltings,et al.  Collusion-resistant, incentive-compatible feedback payments , 2007, EC '07.

[12]  Jens Witkowski Robust peer prediction mechanisms , 2015 .

[13]  Ryan P. Adams,et al.  Trick or treat: putting peer prediction to the test , 2014 .

[14]  Kannan Ramchandran,et al.  Truth Serums for Massively Crowdsourced Evaluation Tasks , 2015, ArXiv.

[15]  R. L. Winkler Scoring Rules and the Evaluation of Probability Assessors , 1969 .

[16]  David C. Parkes,et al.  Learning the Prior in Minimal Peer Prediction , 2013 .

[17]  David C. Parkes,et al.  Peer prediction without a common prior , 2012, EC '12.

[18]  Boi Faltings,et al.  Mechanisms for Making Crowds Truthful , 2014, J. Artif. Intell. Res..

[19]  David C. Parkes,et al.  A Robust Bayesian Truth Serum for Small Populations , 2012, AAAI.

[20]  Paul Resnick,et al.  Eliciting Informative Feedback: The Peer-Prediction Method , 2005, Manag. Sci..

[21]  Anirban Dasgupta,et al.  Crowdsourced judgement elicitation with endogenous proficiency , 2013, WWW.

[22]  D. Prelec A Bayesian Truth Serum for Subjective Data , 2004, Science.

[23]  Boi Faltings,et al.  Incentive Schemes for Participatory Sensing , 2015, AAMAS.