Finite state stochastic games: Existence theorems and computational procedures

Let {X_{n}} be a Markov process with finite state space and transition probabilities p_{ij}(u_{i}, v_{i}) depending on u i and v_{i}. State 0 is the capture state (where the game ends; p_{oi} \equiv \delta_{oi}) ; u = {u_{i}} and v = {v_{i}} are the pursuer and evader strategies, respectively, and are to be chosen so that capture is advanced or delayed and the cost C_{i^{u,v}} = E[\Sum_{0}^{\infty} k (u(X_{n}), v(X_{n}), X_{n}) | X_{0} = i] is minimaxed (or maximined), where k(\alpha, \beta, 0) \equiv 0 . The existence of a saddle point and optimal strategy pair or e-optimal strategy pair is considered under several conditions. Recursive schemes for computing the optimal or e-optimal pairs are given.