Universality of sparse d > 2 conformal field theory at large N