Graphs, dynamic programming, and finite games