Constraints on planet formation via gravitational instability across cosmic time