Constraints on dark matter annihilation to fermions and a photon