The power of threats in stochastic games

In the theory of limiting average reward infinitely repeated games the Folk theorem tells us that any feasible and individually rational reward can be achieved as an equilibrium reward. The standard proof of this theorem involves pure strategies that yield this reward and threats to prevent the opponent from deviating from his pure strategy. In stochastic games it is not always possible to apply threats in a similar fashion, since a deviation may take play to a different state at which punishment is ineffective. Nevertheless, threats allow us to formulate sufficient, and quite general, conditions for the existence of limiting average ∈-equilibria.