Discrete stop-or-go games

Dubins and Savage (How to gamble if you must: inequalities for stochastic processes, McGraw-Hill, New York, 1965) found an optimal strategy for limsup gambling problems in which a player has at most two choices at every state x at most one of which could differ from the point mass $$\delta (x)$$ δ ( x ) . Their result is extended here to a family of two-person, zero-sum stochastic games in which each player is similarly restricted. For these games we show that player 1 always has a pure optimal stationary strategy and that player 2 has a pure $$\epsilon $$ ϵ -optimal stationary strategy for every $$\epsilon > 0$$ ϵ > 0 . However, player 2 has no optimal strategy in general. A generalization to n-person games is formulated and $$\epsilon $$ ϵ -equilibria are constructed.