Distributional and hierarchical reinforcement learning for physical systems with noisy state observations and exogenous perturbations