Equilibrium in a stochastic $n$-person game

Heuristically, a stochastic game is described by a sequence of states which are determined stochastically. The stochastic element arises from a set of transition probability measures. The determination of the particular transition probability measure to be used at a move of the game is controlled in part by each of the n players and it is this determination scheme which gives rise to the strategies. One might consider the following economics problem. We have n firms competing for a market. Each will make a strategy decision periodically. During each period the n firms play an ?z-person game. Across the infinite horizon then we have a sequence of games. The economic situation behaves in such a way that the game played in a given period depends on the game played in the previous period and the strategies used in this period. The dependence is not deterministic but is stochastic. A player's strategy will reflect a concern both for the game now being played and for the situation that will be probably confronted in the next period. The relative strength of these two concerns will be in a geometric ratio, the so-called discounting over the infinite horizon. We shall call the outcome of each game a cost to each player. Negative cost is thus a gain. More precisely we are considering a game which is described by a finite set of states /. A play is a sequence of states {/»}£= 0> &' » 6 1 Ή the game is in the state /, each player h may choose an alternative jh eJ (Γ). This choice is made with knowledge of the state ί. Once each player has made his choice, the game proceeds to the state fe with probability PH^.J^ Here Fίy1...;w^>0 and ^Pijr..jnk = 1. The cost to player h of being in state ί and having the vector ; = (/ι, ••-, /Λ) chosen is ChΓj. Each player furthermore chooses to discount his projected cost by a factor αΛ, where 0 < α Λ < l . Thus if a sequence of states {ίn} and alternative choices {]n} have been made, the cost to player h is given by