Peer-Prediction in the Presence of Outcome Dependent Lying Incentives

We derive conditions under which a peerconsistency mechanism can be used to elicit truthful data from non-trusted rational agents when an aggregate statistic of the collected data affects the amount of their incentives to lie. Furthermore, we discuss the relative saving that can be achieved by the mechanism, compared to the rational outcome, if no such mechanism was implemented. Our work is motivated by distributed platforms, where decentralized data oracles collect information about realworld events, based on the aggregate information provided by often self-interested participants. We compare our theoretical observations with numerical simulations on two public real datasets.

[1]  Barteld Kooi,et al.  Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems , 2011, Adaptive Agents and Multi-Agent Systems.

[2]  Boi Faltings,et al.  Mechanisms for Making Crowds Truthful , 2014, J. Artif. Intell. Res..

[3]  A. Rubinstein.  Modeling Bounded Rationality , 1998 .

[4]  Kevin Leyton-Brown,et al.  Incentivizing Evaluation via Limited Access to Ground Truth: Peer-Prediction Makes Things Worse , 2016, ArXiv.

[5]  Boi Faltings,et al.  Incentive Schemes for Participatory Sensing , 2015, AAMAS.

[6]  Yiling Chen,et al.  Output Agreement Mechanisms and Common Knowledge , 2014, HCOMP.

[7]  Grant Schoenebeck,et al.  Equilibrium Selection in Information Elicitation without Verification via Information Monotonicity , 2018, ITCS.

[8]  L. Fortnow,et al.  Proceedings of the 10th ACM conference on Electronic commerce , 2009, EC 2009.

[9]  Boi Faltings,et al.  Incentives for Effort in Crowdsourcing Using the Peer Truth Serum , 2016, ACM Trans. Intell. Syst. Technol..

[10]  Virgílio A. F. Almeida,et al.  Proceedings of the 22nd international conference on World Wide Web , 2013, WWW 2013.

[11]  David M. Pennock,et al.  Crowdsourced Outcome Determination in Prediction Markets , 2017, AAAI.

[12]  Ian A. Kash,et al.  Market manipulation with outside incentives , 2011, Autonomous Agents and Multi-Agent Systems.

[13]  Moshe Babaioff,et al.  Proceedings of the 2017 ACM Conference on Economics and Computation , 2017, EC.

[14]  Zibin Zheng,et al.  Investigating QoS of Real-World Web Services , 2014, IEEE Transactions on Services Computing.

[15]  Arpit Agarwal,et al.  Informed Truthfulness in Multi-Task Peer Prediction , 2016, EC.

[16]  Peter F. Patel-Schneider,et al.  Proceedings of the 16th international conference on World Wide Web , 2007, WWW 2007.

[17]  Boi Faltings,et al.  A Robust Bayesian Truth Serum for Non-Binary Signals , 2013, AAAI.

[18]  Paul Resnick,et al.  Eliciting Informative Feedback: The Peer-Prediction Method , 2005, Manag. Sci..

[19]  Arpit Agarwal,et al.  Peer Prediction with Heterogeneous Users , 2017, EC.

[20]  Boi Faltings,et al.  Deep Bayesian Trust: A Dominant and Fair Incentive Mechanism for Crowd , 2018, AAAI.

[21]  Sanmay Das,et al.  Trading on a Rigged Game: Outcome Manipulation in Prediction Markets , 2016, IJCAI.

[22]  Boi Faltings,et al.  Game Theory for Data Science: Eliciting Truthful Information , 2017, Game Theory for Data Science.

[23]  Kannan Ramchandran,et al.  Truth Serums for Massively Crowdsourced Evaluation Tasks , 2015, ArXiv.

[24]  L. Christophorou.  Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[25]  Boi Faltings,et al.  Incentives for Answering Hypothetical Questions , 2011 .