Micro- and Macro-Level Validation in Agent-Based Simulation: Reproduction of Human-Like Behaviors and Thinking in a Sequential Bargaining Game

This paper addresses both micro- and macro-level validation in agent-based simulation (ABS) to explore validated agents that can reproduce not only human-like behaviors externally but also human-like thinking internally. For this purpose, we employ the sequential bargaining game, which can investigate a change in humans' behaviors and thinking longer than the ultimatum game (i.e., one-time bargaining game), and compare simulation results of Q-learning agents employing any type of the three types of action selections (i.e., the e-greedy, roulette, and Boltzmann distribution selections) in the game. Intensive simulations have revealed the following implications: (1) Q-learning agents with any type of three action selections can reproduce human-like behaviors but not human-like thinking, which means that they are validated from the macro-level viewpoint but not from the micro-level viewpoint; and (2) Q-learning agents employing Boltzmann distribution selection with changing the random parameter can reproduce both human-like behaviors and thinking, which means that they are validated from both micro- and macro-level viewpoints.

[1]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[2]  Abhinay Muthoo,et al.  A Non-Technical Introduction to Bargaining Theory , 2000 .

[3]  Jaime Simão Sichman,et al.  Multi-Agent-Based Simulation , 2002, Lecture Notes in Computer Science.

[4]  Alvin E. Roth,et al.  Modelling Predicting How People Play Games: Reinforcement learning in experimental games with unique , 1998 .

[5]  A. Rubinstein.  Perfect Equilibrium in a Bargaining Model , 1982 .

[6]  Steven Durlauf,et al.  Growing artificial societies , 1997, Complex.

[7]  Joshua M. Epstein,et al.  Growing artificial societies , 1996 .

[8]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[9]  Sander van der Hoog,et al.  On Multi-Agent Based Simulation , 2004 .

[10]  A. Roth,et al.  Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term* , 1995 .

[11]  R. Axelrod,et al.  The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration , 1998 .

[12]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[13]  Keiki Takadama,et al.  Can Agents Acquire Human-Like Behaviors in a Sequential Bargaining Game? - Comparison of Roth's and Q-Learning Agents - , 2006, MABS.

[14]  A. Muthoo.  Bargaining Theory with Applications , 1999 .

[15]  Daniel Friedman,et al.  Experimental Methods: A Primer for Economists , 1994 .

[16]  Makoto Yokoo,et al.  Reinforcement Learning on Monopolistic Intermediary Games: Subject Experiments and Simulation , 2007 .

[17]  G. Owen,et al.  Two-person bargaining: An experimental test of the Nash axioms , 1974 .

[18]  B. Scholtens,et al.  Market microstructure : Intermediaries and the theory of the firm. , 2000 .

[19]  T. Schelling,et al.  The Strategy of Conflict. , 1961 .

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Norikazu Sugimoto,et al.  Cross-Element Validation in Multiagent-based Simulation: Switching Learning Mechanisms in Agents , 2003, J. Artif. Soc. Soc. Simul..

[22]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[23]  Bruce Edmonds,et al.  Model-to-Model Analysis , 2003 .

[24]  J. Kagel,et al.  Handbook of Experimental Economics , 1997 .

[25]  Robert L. Axtell,et al.  Aligning simulation models: A case study and results , 1996, Comput. Math. Organ. Theory.

[26]  G. Nigel Gilbert,et al.  Open Problems in Using Agent-Based Models in Industrial and Labor Dynamics , 2004, Adv. Complex Syst..

[27]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[28]  T. Bosse,et al.  Human vs. computer behavior in multi-issue negotiation , 2005, Rational, Robust, and Secure Negotiation Mechanisms in Multi-Agent Systems (RRS'05).

[29]  T. Schelling.  The Strategy of Conflict , 1963 .

[30]  Andrew M. Colman,et al.  The complexity of cooperation: Agent-based models of competition and collaboration , 1998, Complex..

[31]  A. Colman,et al.  The complexity of cooperation: Agent-based models of competition and collaboration , 1998, Complex..

[32]  S. Zamir,et al.  Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study , 1991 .

[33]  R. Inderst.  Bargaining Theory with Applications , 2002 .