An inverse correlated equilibrium framework for utility learning in multiplayer, noncooperative settings

In a game-theoretic framework, given parametric agent utility functions, we solve the inverse problem of computing the feasible set of utility function parameters for each individual agent, given that they play a correlated equilibrium strategy. We model agents as utility maximizers, then cast the problem of computing the parameters of players' utility functions as a linear program using the fact that their play results in a correlated equilibrium. We focus on situations where agents must make tradeoffs between multiple competing components within their utility function. We test our method first on a simulated game of Chicken-Dare, and then on data collected in a real-world trial of a mobile fitness game in which five players must balance between protecting their privacy and receiving a reward for burning calories and improving their physical fitness. Through the learned utility functions from the fitness game, we hope to gain insight into the relative importance each user places on safeguarding their privacy vs. achieving the other desirable objectives in the game.

[1]  Christos Dimitrakakis,et al.  Preference elicitation and inverse reinforcement learning , 2011, ECML/PKDD.

[2]  Luis E. Ortiz,et al.  Maximum Entropy Correlated Equilibria , 2007, AISTATS.

[3]  C. Papadimitriou Algorithmic Game Theory: The Complexity of Finding Nash Equilibria , 2007 .

[4]  Asuman E. Ozdaglar,et al.  Correlated equilibria in continuous games: Characterization and computation , 2008, Games Econ. Behav..

[5]  Dean P. Foster,et al.  Regret in the On-Line Decision Problem , 1999 .

[6]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[7]  Bradley Malin,et al.  Towards the security and privacy analysis of patient portals , 2007, SIGBED.

[8]  Radha Poovendran,et al.  Protecting patient privacy against unauthorized release of medical images in a group communication environment. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[9]  R. Aumann Subjectivity and Correlation in Randomized Strategies , 1974 .

[10]  Gábor Lugosi,et al.  Learning correlated equilibria in games with compact sets of strategies , 2007, Games Econ. Behav..

[11]  Ulrike von Luxburg,et al.  Proceedings of the 28th International Conference on Machine Learning, ICML 2011 , 2011, International Conference on Machine Learning.

[12]  Christos H. Papadimitriou,et al.  Computing correlated equilibria in multi-player games , 2005, STOC '05.

[13]  Kevin Waugh,et al.  Computational Rationalization: The Inverse Equilibrium Problem , 2011, ICML.

[14]  Y. Mansour,et al.  Algorithmic Game Theory: Learning, Regret Minimization, and Equilibria , 2007 .

[15]  E Ammenwerth,et al.  Model-based Design of Trustworthy Health Information Systems , 2008, Methods of Information in Medicine.

[16]  Charles L. Isbell,et al.  Markov Games of Incomplete Information for Multi-Agent Reinforcement Learning , 2011, Interactive Decision Theory and Game Theory.

[17]  S. Hart,et al.  Uncoupled Dynamics Do Not Lead to Nash Equilibrium , 2003 .

[18]  Anne Miller,et al.  A Model-Integrated, Guideline-Driven, Clinical Decision-Support System , 2009, IEEE Software.

[19]  R. Aumann Correlated Equilibrium as an Expression of Bayesian Rationality Author ( s ) , 1987 .