Convergence properties of optimal control algorithms