Computing Stackelberg Equilibria in Discounted Stochastic Games

Stackelberg games increasingly influence security policies deployed in real-world settings. Much of the work to date focuses on devising a fixed randomized strategy for the defender, accounting for an attacker who optimally responds to it. In practice, defense policies are often subject to constraints and vary over time, allowing an attacker to infer characteristics of future policies based on current observations. A defender must therefore account for an attacker's observation capabilities in devising a security policy. We show that this general modeling framework can be captured using stochastic Stackelberg games (SSGs), where a defender commits to a dynamic policy to which the attacker devises an optimal dynamic response. We then offer the following contributions. 1) We show that Markov stationary policies suffice in SSGs, 2) present a finite-time mixedinteger non-linear program for computing a Stackelberg equilibrium in SSGs, and 3) present a mixed-integer linear program to approximate it. 4)We illustrate our algorithms on a simple SSG representing an adversarial patrolling scenario, where we study the impact of attacker patience and risk aversion on optimal defense policies.

[1]  Garth P. McCormick Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[2]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[3]  C. Gollier The economics of risk and time , 2001 .

[4]  Vincent Conitzer,et al.  Computing the optimal strategy to commit to , 2006, EC '06.

[5]  Playing games for security: an efficient exact algorithm for solving Bayesian Stackelberg games , 2008, AAMAS 2008.

[6]  Sarit Kraus,et al.  Multi-robot perimeter patrol in adversarial settings , 2008, 2008 IEEE International Conference on Robotics and Automation.

[7]  Manish Jain,et al.  Computing optimal randomized resource allocations for massive security games , 2009, AAMAS.

[8]  Nicola Basilico,et al.  Leader-follower strategies for robotic patrolling in environments with arbitrary topologies , 2009, AAMAS.

[9]  Manish Jain,et al.  Software Assistants for Randomized Patrol Planning for the LAX Airport Police and the Federal Air Marshal Service , 2010, Interfaces.

[10]  Tuomas Sandholm,et al.  Computing equilibria by incorporating qualitative models? , 2010, AAMAS.

[11]  Nicola Basilico,et al.  Asynchronous Multi-Robot Patrolling against Intrusions in Arbitrary Topologies , 2010, AAAI.

[12]  Manish Jain,et al.  Security Games with Arbitrary Schedules: A Branch and Price Approach , 2010, AAAI.

[13]  Nicola Basilico,et al.  A Game-Theoretical Model Applied to an Active Patrolling Camera , 2010, 2010 International Conference on Emerging Security Technologies.

[14]  Noa Agmon,et al.  Multiagent Patrol Generalized to Complex Environmental Conditions , 2011, AAAI.

[15]  Nicola Basilico,et al.  Automated Abstractions for Patrolling Security Games , 2011, AAAI.

[16]  Branislav Bosanský,et al.  Computing time-dependent policies for patrolling games with mobile targets , 2011, AAMAS.

[17]  Bo An,et al.  GUARDS and PROTECT: next generation applications of security games , 2011, SECO.

[18]  Bo An,et al.  Adversarial patrolling games , 2012, AAMAS.