Average Optimality in Markov Games with General State Space

A Markov game with general state space and the average reward as opti-mality criterion is considered. Asymmetric conditions are presented under which the game has a value and both player have-optimal stationary policies. We also study the convergence of the value iteration and the linear programming formulation. It is shown that the associated linear programs are solvable and that there is no duality gap.