Infinite-horizon average-cost Markov decision process routing games

We explore an extension of nonatomic routing games that we call Markov decision process routing games where each agent chooses a transition policy between nodes in a network rather than a path from an origin node to a destination node, i.e. each agent in the population solves a Markov decision process rather than a shortest path problem. This type of game was first introduced in [1] in the finite-horizon total-cost case. Here we present the infinite-horizon average-cost case. We present the appropriate definition of a Wardrop equilibrium as well as a potential function program for finding the equilibrium. This work can be thought of as a routing-game-based formulation of continuous population stochastic games (mean-field games or anonymous sequential games). We apply our model to ridesharing drivers competing for fares in an urban area.

[1]  William H. Sandholm,et al.  Potential Games with Continuous Player Sets , 2001, J. Econ. Theory.

[2]  P. Lions,et al.  Jeux à champ moyen. II – Horizon fini et contrôle optimal , 2006 .

[3]  P. Lions,et al.  Mean field games , 2007 .

[4]  C. B. Mcguire,et al.  Studies in the Economics of Transportation , 1958 .

[5]  Olivier Guéant.  Existence and Uniqueness Result for Mean Field Games with Congestion Effect on Graphs , 2011, 1110.3442.

[6]  Dan Bernhardt,et al.  Anonymous Sequential Games with General State Space , 1991 .

[7]  Eitan Altman,et al.  Stochastic State Dependent Population Games in Wireless Communication , 2011, IEEE Transactions on Automatic Control.

[8]  Eitan Altman,et al.  An Anonymous Sequential Game Approach for Battery State Dependent Power Control , 2009, NET-COOP.

[9]  Alan S. Manne.  Linear Programming and Sequential Decision Models , 1959 .

[10]  R. Rosenthal,et al.  Anonymous sequential games , 1988 .

[11]  D. Bernhardt,et al.  Anonymous sequential games: Existence and characterization of equilibria , 1995 .

[12]  Stella C. Dafermos,et al.  Traffic assignment problem for a general network , 1969 .

[13]  Michael Patriksson,et al.  The Traffic Assignment Problem: Models and Methods , 2015 .

[14]  P. Lions,et al.  Jeux à champ moyen. I – Le cas stationnaire , 2006 .

[15]  Eitan Altman,et al.  Stationary Anonymous Sequential Games with Undiscounted Rewards , 2011, Journal of Optimization Theory and Applications.

[16]  R. B. Kulkarni,et al.  Linear programming formulations of Markov decision processes , 1986 .

[17]  D. Gomes,et al.  Discrete Time, Finite State Space Mean Field Games , 2010 .

[18]  J. Wardrop.  ROAD PAPER. SOME THEORETICAL ASPECTS OF ROAD TRAFFIC RESEARCH. , 1952 .

[19]  S. Shankar Sastry,et al.  Markov Decision Process Routing Games , 2017, 2017 ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS).

[20]  Olivier Guéant.  From infinity to one: The reduction of some mean field games to a global control problem , 2011, 1110.3441.