Probably Almost Stable Strategy Profiles in Simulation-Based Games

Empirical studies of strategic settings commonly model player interactions under supposed game-theoretic equilibrium behavior, to predict what rational agents might do. But in sufficiently complex settings, analysts cannot solve for exact equilibria, and may resort to solving a restricted game where agents are limited to a tractable subset of strategies. This provides a solution, but one with unclear strategic stability in the original game. We propose a search and evaluation method that can guarantee a well-defined strategic stability property in the profile that it yields, even if only a small subset of possible strategies in a game have been analyzed. The method achieves this result by combining statistical confidence interval estimation, a multiple test correction, and empirical game-theoretic analysis. We also present an extension of the method that more often finds genuine approximate equilibria, by using simulated annealing instead of simple random search for strategy exploration. We demonstrate efficacy in two example settings: the first-price sealed-bid auction, and a cybersecurity game.