Artificial agents learning human fairness

Recent advances in technology allow multi-agent systems to be deployed in cooperation with or as a service for humans. Typically, those systems are designed assuming individually rational agents, according to the principles of classical game theory. However, research in the field of behavioral economics has shown that humans are not purely self-interested: they strongly care about fairness. Therefore, multi-agent systems that fail to take fairness into account, may not be sufficiently aligned with human expectations and may not reach intended goals. In this paper, we present a computational model for achieving fairness in adaptive multi-agent systems. The model uses a combination of Continuous Action Learning Automata and the Homo Egualis utility function. The novel contribution of our work is that this function is used in an explicit, computational manner. We show that results obtained by agents using this model are compatible with experimental and analytical results on human fairness, obtained in the field of behavioral economics.

[1]  E. Fehr,et al.  Fairness and Retaliation: The Economics of Reciprocity , 2000, SSRN Electronic Journal.

[2]  Armin Falk,et al.  A Theory of Reciprocity , 2001, Games Econ. Behav..

[3]  Huib Aldewereld,et al.  Autonomy vs. Conformity: An Institutional Perspective on Norms and Protocols , 2007, The Knowledge Engineering Review.

[4]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[5]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[6]  Herbert Gintis,et al.  Game Theory Evolving: A Problem-Centered Introduction to Modeling Strategic Interaction - Second Edition , 2009 .

[7]  E. Fehr.  A Theory of Fairness, Competition and Cooperation , 1998 .

[8]  J. Nash.  THE BARGAINING PROBLEM , 1950, Classics in Game Theory.

[9]  H. Oosterbeek,et al.  Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis , 2001 .

[10]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[11]  Nico Roos,et al.  Agent-based scheduling for aircraft deicing , 2006 .

[12]  Sandip Sen,et al.  Emergence of Norms through Social Learning , 2007, IJCAI.

[13]  W. Güth,et al.  An experimental analysis of ultimatum bargaining , 1982 .

[14]  M. Milinski,et al.  Reputation helps solve the ‘tragedy of the commons’ , 2002, Nature.

[15]  C. Hauert,et al.  Via Freedom to Coercion: The Emergence of Costly Punishment , 2007, Science.

[16]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[17]  H. Simon,et al.  Models of Man. , 1957 .

[18]  F. C. Santos,et al.  Evolutionary dynamics of social dilemmas in structured heterogeneous populations. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[19]  A. Roth,et al.  Game-Theoretic Models and the Role of Information in Bargaining , 1979 .

[20]  Stuart J. Russell,et al.  Artificial Intelligence , 1999 .

[21]  Francisco C. Santos,et al.  Cooperation Prevails When Individuals Adjust Their Social Ties , 2006, PLoS Comput. Biol..

[22]  E. Fehr,et al.  Altruistic punishment in humans , 2002, Nature.

[23]  Kaushik Basu,et al.  The traveler's dilemma. , 2007, Scientific American.

[24]  Ida G. Sprinkhuizen-Kuyper,et al.  Robust and Scalable Coordination of Potential-Field Driven Agents , 2006, 2006 International Conference on Computational Inteligence for Modelling Control and Automation and International Conference on Intelligent Agents Web Technologies and International Commerce (CIMCA'06).

[25]  K. Taylor.  Natural justice. , 1998, The Lamp.

[26]  M. Nowak,et al.  Fairness versus reason in the ultimatum game. , 2000, Science.

[27]  M. Thathachar,et al.  Networks of Learning Automata: Techniques for Online Stochastic Optimization , 2003 .

[28]  Ernst Fehr,et al.  Homo reciprocans: A Research Initiative on the Origins, Dimensions, and Policy Implications of Recip , 1997 .

[29]  Ann Nowé,et al.  Exploring selfish reinforcement learning in repeated games with stochastic rewards , 2007, Autonomous Agents and Multi-Agent Systems.

[30]  M. Rabin,et al.  Understanding Social Preference with Simple Tests , 2001 .

[31]  David C. Parkes,et al.  Computational Mechanism Design , 2007 .

[32]  C. Hauert,et al.  Reward and punishment , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Yann Chevaleyre,et al.  Issues in Multiagent Resource Allocation , 2006, Informatica.

[34]  Nico Roos,et al.  Priority Awareness: Towards a Computational Model of Human Fairness for Multi-agent Systems , 2007, Adaptive Agents and Multi-Agents Systems.

[35]  T. Yamagishi.  The provision of a sanctioning system as a public good , 1986 .

[36]  C. Hauert,et al.  Volunteering as Red Queen Mechanism for Cooperation in Public Goods Games , 2002, Science.

[37]  Colin Camerer,et al.  Foundations of Human Sociality - Economic Experiments and Ethnographic: Evidence From Fifteen Small-Scale Societies , 2004 .

[38]  Javier Vázquez-Salceda,et al.  Norms in multiagent systems: from theory to practice , 2005, Comput. Syst. Sci. Eng..

[39]  Klaus M. Schmidt,et al.  A Theory of Fairness, Competition, and Cooperation , 1999 .

[40]  Juan Antonio Rodríguez-Aguilar,et al.  On the design and construction of agent-mediated institutions , 2001 .

[41]  Yoav Shoham,et al.  If multi-agent learning is the answer, what is the question? , 2007, Artif. Intell..

[42]  Yann Chevaleyre,et al.  A Short Introduction to Computational Social Choice , 2007, SOFSEM.

[43]  E. Fehr.  Human behaviour: Don't lose your reputation , 2004, Nature.

[44]  R. Boyd,et al.  Indirect reciprocity can stabilize cooperation without the second-order free rider problem , 2004, Nature.

[45]  Manuela M. Veloso,et al.  Multiagent learning using a variable learning rate , 2002, Artif. Intell..

[46]  Thomas Riechmann,et al.  Inequity Aversion and Individual Behavior in Public Good Games: An Experimental Investigation , 2007 .

[47]  G. Owen,et al.  Two-person bargaining: An experimental test of the Nash axioms , 1974 .

[48]  Ann Nowé,et al.  Evolutionary game theory and multi-agent reinforcement learning , 2005, The Knowledge Engineering Review.