Cosmological constraints on ultra-light axion fields