Dantzig's pivoting rule for shortest paths, deterministic MDPs, and minimum cost to time ratio cycles

Dantzig's pivoting rule is one of the most studied pivoting rules for the simplex algorithm. While the simplex algorithm with Dantzig's rule may require an exponential number of pivoting steps on general linear programs, and even on min cost flow problems, Orlin showed that O(mn2 log n) Dantzig's pivoting steps suffice to solve shortest paths problems, where n and m are the number of vertices and edges, respectively, in the graph. Post and Ye recently showed that the simplex algorithm with Dantzig's rule requires only O(m2n3 log2 n) pivoting steps to solve deterministic MDPs with the same discount factor for each edge, and only O(m3n5 log2 n) pivoting steps to solve deterministic MDPs with possibly a distinct discount factor for each edge. We improve Orlin's bound for shortest paths and Post and Ye's bound for deterministic MDPs with the same discount factor by a factor of n to O(mn log n), and O(m2n2 log2 n), respectively. We also improve by a factor of n the bound for deterministic MDPs with varying discounts when all discount factors are sufficiently close to 1. These bounds follow from a new proof technique showing that after a certain number of steps, either many edges are excluded from participating in further policies, or there is a large decrease in the value. We also obtain an Ω(n2) lower bound on the number of Dantzig's pivoting steps required to solve shortest paths problems, even when m = Θ(n). Finally, we describe a reduction from the problem of finding a minimum cost to time ratio cycle to the problem of finding an optimal policy for a discounted deterministic MDP with varying discount factors that tend to 1. This gives a strongly polynomial time algorithm for the problem that does not use Megiddo's parametric search technique.

[1]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[2]  Yinyu Ye,et al.  The Simplex Method is Strongly Polynomial for Deterministic Markov Decision Processes , 2012, Math. Oper. Res..

[3]  Nimrod Megiddo,et al.  Applying parallel computation algorithms in the design of serial algorithms , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[4]  J. Orlin On the simplex algorithm for networks and generalized networks , 1983 .

[5]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[6]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[7]  N. Amenta,et al.  Deformed products and maximal shadows of polytopes , 1996 .

[8]  Cyrus Derman,et al.  Finite State Markovian Decision Processes , 1970 .

[9]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[10]  V. Klee,et al.  HOW GOOD IS THE SIMPLEX ALGORITHM , 1970 .

[11]  Bernd Gärtner,et al.  Understanding and using linear programming , 2007, Universitext.

[12]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[13]  Ravindra K. Ahuja,et al.  Network Flows: Theory, Algorithms, and Applications , 1993 .

[14]  Mikkel Thorup,et al.  Discounted deterministic Markov decision processes and discounted all-pairs shortest paths , 2009, TALG.

[15]  Norman Zadeh,et al.  A bad network problem for the simplex method and other minimum cost flow algorithms , 1973, Math. Program..

[16]  T. Lindvall ON A ROUTING PROBLEM , 2004, Probability in the Engineering and Informational Sciences.

[17]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[18]  Richard M. Karp,et al.  A characterization of the minimum cycle mean in a digraph , 1978, Discret. Math..

[19]  L. R. Ford,et al.  NETWORK FLOW THEORY , 1956 .

[20]  Nesa L'abbe Wu,et al.  Linear programming and extensions , 1981 .

[21]  G. Dantzig,et al.  FINDING A CYCLE IN A GRAPH WITH MINIMUM COST TO TIME RATIO WITH APPLICATION TO A SHIP ROUTING PROBLEM , 1966 .

[22]  Yinyu Ye,et al.  The Simplex and Policy-Iteration Methods Are Strongly Polynomial for the Markov Decision Problem with a Fixed Discount Rate , 2011, Math. Oper. Res..

[23]  James B. Orlin,et al.  A polynomial time primal network simplex algorithm for minimum cost flows , 1996, SODA '96.