Randomized function fitting-based empirical value iteration

Randomization is notable for being much less computationally expensive than optimization but often yielding comparable numerical performance. In this paper, we consider randomized function fitting combined with empirical value iteration for approximate dynamic programming on continuous state spaces. The method we propose is universal (i.e., not problem-dependent) and yields good approximations with high probability. A random operator theoretic framework is introduced for convergence analysis which uses a novel stochastic dominance argument. A non-asymptotic rate of convergence is obtained as a byproduct of the analysis. Numerical experiments confirm good performance of the algorithm proposed.