SU(4) flavor symmetry breaking in D-meson couplings to light hadrons