Probabilistic Model Checking Multi-agent Behaviors in Dispersion Games Using Counter Abstraction

Accurate analysis of the stochastic dynamics of multi-agent system is important but challenging. Probabilistic model checking, a formal technique for analysing a system which exhibits stochastic behaviors, can be a natural solution to analyse multi-agent systems. In this paper, we investigate this problem in the context of dispersion games focusing on two strategies: basic simple strategy (BSS) and extended simple strategies (ESS). We model the system using discrete-time Markov chain (DTMC) and reduce the state space of the models by applying counter abstraction technique. Two important properties of the system are considered: convergence and convergence rate. We show that these kinds of properties can be automatically analysed and verified using probabilistic model checking techniques. Better understanding of the dynamics of the strategies can be obtained compared with empirical evaluations in previous work. Through the analysis, we are able to demonstrate that probabilistic model checking technique is applicable, and indeed useful for automatic analysis and verification of multi-agent dynamics.

[1]  Walter Van de Velde,et al.  Agents Breaking Away , 1996, Lecture Notes in Computer Science.

[2]  Ana Cavalcanti,et al.  FM 2009: Formal Methods, Second World Congress, Eindhoven, The Netherlands, November 2-6, 2009. Proceedings , 2009, FM.

[3]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[4]  Jun Sun,et al.  Fair Model Checking with Process Counter Abstraction , 2009, FM.

[5]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[6]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[7]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[8]  Tom Lenaerts,et al.  A selection-mutation model for q-learning in multi-agent systems , 2003, AAMAS '03.

[9]  Axel Legay,et al.  Statistical Model Checking: An Overview , 2010, RV.

[10]  Wesley Kerr,et al.  Safety and Security in Multiagent Systems - Research Results from 2004-2006 , 2009, Safety and Security in Multiagent Systems.

[11]  Ryszard Kowalczyk,et al.  Dynamic analysis of multiagent {\it Q}-learning with {\&}epsilon;-greedy exploration , 2009, ICML 2009.

[12]  Klaus Havelund,et al.  Model checking programs , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[13]  Jun Sun,et al.  Analyzing multi-agent systems with probabilistic model checking approach , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[14]  Michael Wooldridge,et al.  Model checking multi-agent systems with MABLE , 2002, AAMAS '02.

[15]  Yi-Cheng Zhang,et al.  Emergence of cooperation and organization in an evolutionary game , 1997 .

[16]  Wamberto Weber Vasconcelos,et al.  Abstractions for model-checking game-theoretic properties of auctions , 2008, AAMAS.

[17]  Ajitha Rajan,et al.  Requirements Coverage as an Adequacy Measure for Conformance Testing , 2008, ICFEM.

[18]  震 中島.  The SPIN Model Checker, G.J.Holzmann著 , 2004 .

[19]  Michael Wooldridge,et al.  Uncertain Agent Verification through Probabilistic Model-Checking , 2009, Safety and Security in Multiagent Systems.

[20]  Jun Sun,et al.  PAT: Towards Flexible Verification under Fairness , 2009, CAV.

[21]  Rafael Heitor Bordini,et al.  Verifying Multi-agent Programs by Model Checking , 2006, Autonomous Agents and Multi-Agent Systems.

[22]  David B. Fogel,et al.  Inductive reasoning and bounded rationality reconsidered , 1999, IEEE Trans. Evol. Comput..

[23]  Andrew Hinton,et al.  PRISM: A Tool for Automatic Verification of Probabilistic Systems , 2006, TACAS.

[24]  Yoav Shoham,et al.  Dispersion games: general definitions and some specific learning results , 2002, AAAI/IAAI.

[25]  Christel Baier,et al.  Principles of model checking , 2008 .

[26]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[27]  A. S. Roa,et al.  AgentSpeak(L): BDI agents speak out in a logical computable language , 1996 .

[28]  Ryszard Kowalczyk,et al.  Dynamic analysis of multiagent Q-learning with ε-greedy exploration , 2009, ICML '09.

[29]  Jun Sun,et al.  Model Checking Hierarchical Probabilistic Systems , 2010, ICFEM.

[30]  Amir Pnueli,et al.  Liveness with (0, 1, ∞)-counter abstraction , 2002 .

[31]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[32]  S. Alpern,et al.  Spatial Dispersion as a Dynamic Coordination Problem , 2002 .

[33]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[34]  Edmund H. Durfee,et al.  Predicting the Expected Behavior of Agents that Learn About Agents: The CLRI Framework , 2004, Autonomous Agents and Multi-Agent Systems.