Noncooperative Differential Games. A Tutorial

These notes provide a brief introduction to the theory of noncooperative differential games. After the Introduction, Section 2 reviews the theory of static games. Different concepts of solution are discussed, including Pareto optima, Nash and Stackelberg equilibria, and the co-co (cooperative-competitive) solutions. Section 3 introduces the basic framework of differential games for two players. Open-loop solutions, where the controls implemented by the players depend only on time, are considered in Section 4. It is shown that Nash and Stackelberg solutions can be computed by solving a two-point boundary value problem for a system of ODEs, derived from the Pontryagin maximum principle. Section 5 deals with solutions in feedback form, where the controls are allowed to depend on time and also on the current state of the system. In this case, the search for Nash equilibrium solutions usually leads to a highly nonlinear system of HamiltonJacobi PDEs. In dimension higher than one, this system is generically not hyperbolic and the Cauchy problem is thus ill posed. Due to this instability, closed-loop solutions to differential games are mainly considered in the special case with linear dynamics and quadratic costs. In Section 6, a game in continuous time is approximated by a finite sequence of static games, by a time discretization. Depending of the type of solution adopted in each static game, one obtains different concept of solutions for the original differential game. Section 7 deals with differential games in infinite time horizon, with exponentially discounted payoffs. In this case, the search for Nash solutions in feedback form leads to a system of time-independent H-J equations. Finally, Section 8 contains a simple example of a game with infinitely many players. This is intended to convey a flavor of the newly emerging theory of mean field games. The Appendix collects background material on multivalued functions, selections and fixed point theorems, optimal control theory and hyperbolic PDEs.