Efficient Algorithms for Learning to Control Bandits with Unobserved Contexts