On the Speed of Convergence of Value Iteration on Stochastic Shortest-Path Problems

We establish a bound on the convergence time of the value iteration algorithm on stochastic shortest-path problems. The bound, which applies for admissible initial vectors as, for example, J\equiv 0 , implies a polynomial-time convergence of value iteration for all problems with polynomially bounded \Vert{J^*}\Vert/\underline{g} . This result gives a partial answer to the open problem of bounding the convergence time of value iteration on arbitrary initial vectors. The proof is obtained by analyzing a stochastic process associated with the shortest-path problem.